See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/351750112

Performance characterization of non-contact infrared thermometers (NCITs)

for forehead temperature measurement

Article  in  Medical Engineering & Physics · May 2021

DOI: 10.1016/j.medengphy.2021.05.007

CITATIONS READS

4 26

7 authors, including:

Nathanael Seay Liang Zhu

University of Maryland, Baltimore County 101 PUBLICATIONS   1,397 CITATIONS   
3 PUBLICATIONS   14 CITATIONS   
SEE PROFILE
SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Medical Countermeasures View project

Development of test methods to evaluate the risk of cancer-cell permeation through tissue containment bags during laparoscopic power morcellation of uterine
fibroids View project

All content following this page was uploaded by Jean E Rinaldi on 20 June 2022.

The user has requested enhancement of the downloaded file.

 Medical Engineering and Physics 93 (2021) 93–99

Contents lists available at ScienceDirect

Medical Engineering and Physics

journal homepage: www.elsevier.com/locate/medengphy

Performance characterization of non-contact infrared thermometers

(NCITs) for forehead temperature measurement
Stacey JL Sullivan a, Nathanael Seay b, Liang Zhu b, Jean E Rinaldi a, Prasanna Hariharan a,
Oleg Vesnovsky a, LD Timmie Topoleski a, b, *
a
Office of Science and Engineering Laboratories, USFDA, 10903 New Hampshire Avenue, Building 62, Silver Spring, MD 20993, United States
b
Department of Mechanical Engineering, UMBC, 1000 Hilltop Circle, ENGR225K, Baltimore, MD 21250, United States

A R T I C L E I N F O A B S T R A C T

Keywords: The ability to assess the performance of a non-contact infrared thermometer (NCIT) may be limited due to the
Thermometer algorithms necessary to predict a reference site temperature (e.g., oral) from a measurement of the forehead skin
Non-contact temperature. The algorithm not only adjusts for the difference between the reference site temperature and
Infrared
forehead temperature, but may also account for hardware corrections, bias adjustments and emissivity settings.
Blackbody
These algorithms are proprietary to the manufacturer and can be unique for each device. ASTM E1965-98 (2016)
Performance
is a standard test method for the evaluation of NCITs. It includes forehead thermometers; however, the algorithm
must be known or an unadjusted calibration mode must be accessible. This study evaluates 6 NCIT models (10
units of each) against the ASTM standard error criterion using a blackbody source. Units were tested within the
manufacturer’s operating and temperature measurement range specification. A method to evaluate measurement
outliers and characterize each model’s performance when the adjustment algorithm is unknown is proposed.
Using this method, 5 of the 6 models had a predicted error > 0.3◦ C.

1. Introduction
Non-contact infrared thermometers (NCITs) used for forehead skin
temperature measurement are widely-accessible and convenient for
determining a reference body site temperature (e.g., oral temperature)
and assessing fevers, particularly with parents and caregivers in the
home. NCITs have gained popularity in pediatrics, especially for infants
and children under five years of age for whom oral or rectal temperature
measurement is invasive, uncomfortable, difficult, time-consuming, and
may be contraindicated [1–3]. In addition, non-contacting temperature
measurement reduces the risk of infectious disease transmission among
individuals [4]. Such design features make the use of NCITs attractive
for screening at ports of entry during infectious disease outbreaks [5].
NCITs derive oral temperature based upon detection of infrared
radiant energy from the surface of the skin. Measurements are taken
most commonly at the forehead, although the temporal artery may also
be used [6]. Temperature measured using an NCIT is affected by the
infrared detector properties, the emissivity of the surface and skin
characteristics, and the ambient environment. The skin surface tem­
perature may deviate several degrees Celsius from the reference body
site temperature (an internal body temperature), especially when the
ambient environment is cold and windy [7].
NCITs include compensation circuitry to correct for detector prop­
erties, surface emissivity, and ambient conditions. In addition, the
manufacturers incorporate an algorithm to adjust for the difference
between the forehead skin surface temperature and the expected target
reference site temperature (the “adjusted mode,” typically referred to as
“subject mode” for most NCITs). Most models also include an unadjusted
mode (“object mode” or “surface mode”). These NCIT software and
design features are proprietary to each manufacturer. NCIT temperature
measurement can also be influenced by the device’s distance and angle
from the target surface and air movement across the target surface.
Thus, the manufacturer provides instructions for the distance from the
surface and the proper positioning to obtain the temperature measure­
ment in a prescribed environment. A feature that is notably missing from
most NCITs is a means for the user to verify if the NCIT is working
properly and providing an accurate measurement.
The accuracy and reproducibility of the temperature measured using
a specific NCIT model or among different models of NCITs are not well-
characterized. Prior to initiating a clinical trial of multiple NCIT models,

* Corresponding author.
E-mail addresses: stacey.sullivan@fda.hhs.gov (S.J. Sullivan), topoleski@umbc.edu (L.T. Topoleski).

https://doi.org/10.1016/j.medengphy.2021.05.007
Received 26 October 2020; Received in revised form 24 April 2021; Accepted 10 May 2021
Available online 21 May 2021
1350-4533/© 2021 Published by Elsevier Ltd on behalf of IPEM.
S.J. Sullivan et al.
we decided to evaluate each model in a laboratory setting. One of the
test methods prescribed in ASTM E1965-98:2016 Standard Specification
for Infrared Thermometers for Intermittent Determination of Patient Tem­
perature is to assess whether an individual NCIT model meets the mea­
surement error acceptance criterion of ± 0.3 ◦ C (± 0.5 ◦ F) when
evaluated within the manufacturer’s specified operating ambient tem­
perature conditions and measurement range using a laboratory black­
body [8]. The standard requires that the temperature adjustment
algorithm be disclosed or the model operates in an optional unadjusted
mode.
The goal of this study was to evaluate the performance of NCITs
using a blackbody source when the adjustment algorithm is unknown by
predicting the offset algorithm and minimizing operator bias. Using a
controlled, uniform, horizontally-mounted infrared blackbody radiation
source (prescribed in ASTM E1965), we assess the in-vitro NCIT mea­
surement variability and reproducibility within a specific NCIT model
and across different models of NCITs when tested in the adjusted mode.
The measurement difference between the NCIT and the blackbody was
determined; however, the error remains unknown because the algorithm
for each model is not accessible. To evaluate the error, a post-processing
and analysis method is proposed to evaluate NCIT performance when
the manufacturer’s algorithm is unknown.
2. Materials and methods
2.1. Procedure
The temperature measurement performance of 60 commercially
available NCITs (6 models, each from a different manufacturer, 10 units
of each model) was evaluated using a commercially available blackbody
source (BBS). All NCITs were purchased from commercial vendors. The
thermometers are designed to display the same temperature that would
be measured by an oral thermometer by measuring the skin temperature
at the center of the forehead (adjusted mode). Ten NCIT sets were
assembled and each NCIT was uniquely identified. Every set contained 6
thermometers, one of each NCIT model (e.g., Set 1 contained device
under test (DUT): 1A, 1B, 1C, 1D, 1E, and 1F). Thermometers were
cleaned and prepared according to the manufacturer’s instructions for
use and had fresh batteries installed prior to testing.
A single BBS (Kambic, Model OB-15/2 BBLT; Semic, Slovenia) was
used to provide a uniform, stable source of infrared radiation to assess
each DUT. The BBS blackbody cavity was horizontally mounted in the
BBS (for measurement access from the side of the BBS, as opposed to the
top of the BBS) and fully immersed in a stirred water bath. The manu­
facturer’s specifications stated stability and uniformity of the bath was
at least ± 0.007◦ C and that the blackbody source conformed to ASTM
E1965-98:2016 Annex A1 [8]. Each NCIT was tested in adjusted mode at
BBS target temperatures (TBBS) of 30, 32, 34, 36, 37, 38, 40, and 42◦ C.
The BBS was set to the target temperature, TBBS, for at least 30 minutes,
until it reached equilibrium as verified by a reference thermometer
(TREF). The reference thermometer was a calibrated NIST traceable
thermistor probe (VRW digital thermometer 89369-140, Radnor, PA).
The reference thermometer measurement accuracy was ±0.05◦ C with a
resolution of 0.0001◦ C and an uncertainty of ± 0.014◦ C for the mea­
surement range of 0◦ C to 100◦ C. Prior to testing, the room temperature
and relative humidity (%RH) were recorded to verify whether the
environmental conditions were within the operating range of all NCIT
models. Measurements for each DUT at a specific TBBS were completed
on the same day by the same operator. All DUTs were stored in the
laboratory where the testing was performed. Eight temperature mea­
surements were collected from each DUT at each TBBS. The reference
temperature, TREF,j, was recorded for every individual temperature
measurement for each DUT (TDUT,j) and used to determine the individual
measurement difference for that DUT (ΔDUT,j) as:
⃒ ⃒
ΔDUT,j = ⃒TDUT,j − TREF,j ⃒ (1)
94
Medical Engineering and Physics 93 (2021) 93–99
where j = 1…8 are the repeated measurements for each TBBS and the
DUT = 1…10 for each model: A, B, C, D, E or F. To allow for the required
rest time between measurements as prescribed by the manufacturer in
the instructions for use, a single temperature measurement was taken by
each DUT in the set (e.g., 1A, 1B, 1C, 1D, 1E, and 1F) before repeating
the process for a total of 8 measurements per DUT per TBBS. During this
time, the ambient room temperature was constant within ±0.5◦ C. The
average difference for a single TBBS for each DUT was then calculated:
(∑ )/
ΔDUTave = ΔDUT,j n (2)
where j = 1…8 are the repeated measurements for each given TBBS, DUT
= 1…10 for each model A, B, C, D, E or F, and n is the total number of
measurements.
The manufacturer’s instructions for use included a measurement
range and its accuracy (Table 1). All NCIT models had a manufacturer’s
specified measurement resolution of 0.1◦ C.
The “measurement range” stated in the manufacturer’s instructions
for use is the range of temperatures for which the NCIT will accurately
measure temperature. An NCIT’s adjusted measurement output is dis­
played using an algorithm to estimate the oral temperature from the
measured forehead skin temperature. The temperature of the forehead
skin is always less than the oral temperature [7], therefore, the NCIT
adjusts the display output by adding to the forehead skin temperature
measurement. The TBBS represents the forehead skin temperature and
not the oral temperature, therefore the TBBS is always lower than the
NCIT display output.
2.2. Measurement exclusions
Any displayed temperature measurement outside of the NCIT man­
ufacturer’s measurement range was excluded from analysis. If more than
half of all measurements for a single model at a single TBBS were
excluded, the TBBS was declared outside of the NCIT’s allowable range
and all were excluded (noted in Table 2 as a “0”). All remaining data sets
comprised a minimum of 6 data points.
The 8th measurement for DUT 1F was discarded due to an incomplete
reference temperature reading.
2.3. Data Post-Processing and Analysis
The original difference calculation (ΔDUT, Eq. (1)) includes the
manufacturer’s temperature offset (Δoffset) as determined by their algo­
rithm. In addition, any error introduced by the difference between the
device emissivity value set by the manufacturer and the actual emis­
sivity of the measured surface (δemissivity) is also present in the DUT
measurement difference, ΔDUT. The remaining portion of ΔDUT is
attributed to measurement error (δDUT) for the DUT.
( )
ΔDUT = Δoffset + δemissivity + δDUT (3)
where:
Δoffset = intentional temperature adjustment by manufacturer’s
algorithm
δemissivity = error due emissivity mismatch [8]
δDUT = measurement errors (imperfect positioning, lens degradation,
etc.)
The algorithm used to determine the adjusted value was not given by
any of the manufacturers, and 5 of the 6 manufacturers did not provide
their NCIT emissivity setting. Although these values are unknown, the
output of each device is still representative of its performance. For each
combination of TBBS and DUT, the BBS set up was the same and room
temperature and relative humidity were considered constant. (All DUTs
were tested within a room temperature range: 21.54◦ C - 24.98◦ C and not
S.J. Sullivan et al. Medical Engineering and Physics 93 (2021) 93–99

Table 1
Manufacturer’s labeling for measurement range and accuracy.
Model A Model B Model C Model D Model E Model F

Adjusted mode accuracy, C ◦

0.2 0.2 0.3 (34.4-35.9), 0.2 (36-39), 0.3 (39.1-42.2) 0.2 0.3 0.3 (34-35.9), 0.2 (36-39), 0.3 (39.1-42.5)
NCIT Measurement Range, ◦ C 34-43 32-42.9 34.4-42.2 34.4-42.2 32-42.5 34-42.5

(∑ )/
Table 2 ΔDUTave,i = ΔDUTi,j ni (6)
Number of data points per NCIT model per TBBS within the manufacturer’s
measurement range.
δDUT,i,j = ΔDUT,i,j − ΔDUTave,i (7)
Model A Model B Model C Model Model E Model F
D ⃒( )⃒
δDUTmax,i = max⃒ δDUT,i,j ⃒ (8)
TBBS ¼ 80 80 80 0 79 76
30◦ C
where j = 1…8 are the repeated measurements for each given TBBS and n
TBBS ¼ 80 80 80 80 80 80
32◦ C = the number of measurements made for a DUT at a specific TBBS.
TBBS ¼ 80 80 80 80 80 80 The average difference is a representation of the repeated measure­
34◦ C ment of a DUT at a specific TBBS (ΔDUTave,i , Eq. (6)). To reduce the in­
TBBS ¼ 80 80 80 80 80 80 fluence of individual δDUT,i,j (Eq. (7))data points suspected of being
36◦ C
TBBS ¼ 80 80 80 80 80 80
anomalous, we calculated the absolute value of the difference between
37◦ C two adjacent data points in the ordered set for a single DUT at a single
TBBS ¼ 80 80 80 0 80 80 TBBS (Fig. 1). If the difference was greater than 0.3◦ C, the data point was
38◦ C removed from the set and the standard deviation of the remaining points
TBBS ¼ 80 0 0 0 80 0
was calculated. If the data point was outside 3 times the standard de­
40◦ C
TBBS ¼ 0 0 0 0 0 0 viation, the point was excluded from the set. If it was within 3 times the
42◦ C standard deviation of the set, the data point was returned to the set. All
Total 560 480 480 320 559 476 remaining data sets comprised a minimum of 6 data points.
The error for a single model can then be characterized by the
maximum absolute difference, δDUTmax, of all DUTs a single model for a
exceeding ±0.5◦ C for each set of measurements. %RH range: 23% –
single TBBS (Eq. (9)).
52%. Note: the %RH for the TBBS = 30◦ C was not taken until the
following day). Given these conditions were constant, the contribution
from the offset algorithm (Δ offset) and the error contribution from the 2.4. Statistical analysis
emissivity setting (δemissivity) remained constant during a single test
period. An analysis of variance (ANOVA) combined with post-hoc pairwise
The initial measurement difference (ΔDUT) is a sum of the intended analysis (Tukey’s Test) was performed. P-values less than 0.05 were
offset value (Δoffset), the error associated with the emissivity mismatch (δ considered statistically significant.
emissivity) and measurement error (δDUT) (Eq. (3)). To characterize the
error of the device, the combined value of the intended offset value and 3. Results
emissivity mismatch (Δoffset + δ emissivity) must be predicted and removed.
The remaining variation can then be used to characterize the measure­ A total of 3,840 temperature measurements were collected, of which
ment error of the DUT (δDUT). 2,875 resulted in temperature performance data that were evaluated.
To obtain the best characterization of the δDUT, the best estimate of The majority of excluded measurements was attributed to falling outside
(Δoffset + δ emissivity) must be calculated. For each data set, ΔDUT,j (Eq. (1)), of the DUT’s measurement range (Table 2). The average difference,
collected for a single DUT at a single TBBS, we assume that (Δoffset + δ ΔDUTave, is shown in Fig. 2 as a function of TBBS.
The test for outliers identified 43 data points > 0.3◦ C from their
emissivity) (from Eq. (3)), is a constant (A) because all influencing factors
(ambient temperature, ambient humidity, TBBS, BBS, power source) are adjacent data point. Of these, 41 were outside of 3 times the standard
kept constant and the remaining variable is the measurement error, deviation of the group and removed from the data set (Table 3).
δDUT. Using the maximum error Eq. (6) across all TBBS for each DUT, 5 of
the 6 NCIT models did not meet the ASTM E1965 criterion specifying
ΔDUT = A + δDUT (4) that the error should be no greater than 0.3◦ C when tested within the
manufacturer’s specified operating ambient conditions over the device’s
where the constant A = (Δ offset + δemissivity) for a single data set (j = 1...8) displayed temperature range [8]; these values are highlighted in orange
∑
and from Eq. (2) (where ΔDUTave = ( ΔDUT,j) /n): in Table 4. Model B met the ASTM criterion both with and without ex­
(∑ ) / clusions applied. However, error values highlighted in purple exceeded
ADUT,i = ΔDUTave,i = ΔDUT,j i ni (5)
their manufacturer’s specification while remaining within the ASTM
criterion (Table 1).
where i is the TBBS (30, 32, 34, 36, 37, 38, 40, and 42◦ C), and n is the To determine if different models reported a different output display
total number of measurements for the specific DUT (e.g., 1A). temperature for the same input temperature (BBS set temp), a pairwise
Therefore, to quantify the individual measurement error (δDUT,i,j, Eq. ∑
comparison between the means of the difference ([ (TDUT-Tref)]/n) of
(7)), the individual measurement difference (ΔDUT,j Eq. (1)), is reduced all 10 samples for each model (A, B, C, D, E, and F) at each specific BBS
by the value of the average difference (ΔDUTave, Eq. (6)) for a single DUT set temperature (TBBS = 30, 32, 34, 36, 37, 38, 40, or 42◦ C) was made
for a single TBBS. The removal of ΔDUTave,i represents the removal of the (Table 5).
offset algorithm and emissivity error contributions. The maximum ab­
solute value of all individual DUT measurement error represents the
4. Discussion
device error (δDUTmax,i, Eq. (8)).
For a specific TBBS (i = 30, 32, 34, 36, 37, 38, 40, and 42◦ C):
This study assessed the performance of consumer-grade NCITs using

95
S.J. Sullivan et al. Medical Engineering and Physics 93 (2021) 93–99

Idenficaon of Outliers
Example: Model A, TBBS = 37°C
Outlier
0.4 adjacent value difference ≤ 0.3°C
δDUTj = ΔDUTj - ΔDUTave, j = 1...8

0.3
0.2
0.1
0
-0.1
-0.2
-0.3
-0.4
-0.5
-0.6
1 2 3 4 5 6 7 8 9 10
DUT Number

Fig. 1. Example of identification of outliers for a single model (model A) at a single TBBS (37◦ C).

7
Average Difference
per NCIT Model per TBBS
Δave = ∑|TDUTj – TBBj| /n, C°

6
A B C D E F
5

0
30 32 34 36 37 38 40
Blackbody Set Temp, °C
Fig. 2. Average difference, for each NCIT model per BBS set temperature, TBBS.

a blackbody source (BBS) in a typical user environment verified to be
within the NCIT operating range. This assessment was carried out to
characterize and understand the performance of each device in a
controlled laboratory setting.
Using the proposed evaluation method, this study found that 5 of the
6 NCIT models did not meet the ASTM acceptance criterion for error. All
laboratory measurements produced a calculated difference (Eq. (1)) that
was outside of the allowable standard error, however this difference
alone cannot be interpreted as non-compliance. This is because every
NCIT was tested in its adjusted mode and therefore every output display
temperature was the measured forehead temperature adjusted to the
oral temperature value using the offset algorithm determined by the
manufacturer. Although the output may not appear accurate (because it
is different from the reference temperature), the measured difference
was a repeatable value with an associated variation. Our post-processing
and analysis isolated this variation to assess the error associated with
repeated temperature measurement.
A limitation of using δDUTmax to estimate error is that it may under­
state or overstate the error value if there was a bias in the measurement.
It is possible that, if the DUT error (δDUT) was consistently biased, the

96
S.J. Sullivan et al. Medical Engineering and Physics 93 (2021) 93–99

Table 3
Data exclusions and outliers.
Model A Model B Model C Model D Model E Model F

Excluded TBBS sets (n¼80); (TDUT outside the 42 C

◦
40 C,
◦
40 C,
◦
30 C,40 C, 42 C
◦ ◦ ◦
42 C
◦
40◦ C, 42◦ C
meas. range.) 42◦ C 42◦ C
Outliers per TBBSDUT; #Excluded| TBBS ¼ 1A; 1|14A; 0| (No data)
#Tested 30◦ C 110A; 2|2
TBBS ¼ 2E; 1|1
• 6 NCIT models 32◦ C
• 10 DUTs per model TBBS ¼ 4D; 1|16D; 2|2
“4A” = sample #4 of model A 34◦ C
TBBS ¼ 2A; 2|29A; 1|1 4D; 1|16D; 2|27D; 1|1 7F; 1|1
36◦ C
TBBS ¼ 3A; 1|17A; 1|1 5C: 1|1 2D; 1|13D; 1|16D; 1|28D; 1| 10E; 1|1
37◦ C 19D; 2|2
TBBS ¼ 1A; 1|1 5A; 1|1 7C; 1|1 (No data) 2E; 1|15E; 1|19E; 3F; 3|3*5F; 2|27F;
38◦ C 1|1 3|3*
TBBS ¼ 2A; 1|16A; 1|1 (No data) (No data) (No data) (No data)
40◦ C
TBBS ¼ (No data) (No data) (No data) (No data) (No data) (No data)
42◦ C
Number of outliers tested 13 0 2 14 5 9
Number of outliers excluded 12 0 2 13 5 9
Data points per modelᵻ 560 480 480 320 559 476
% of excluded outliers 2.1% 0.0% 0.4% 4.1% 0.9% 1.9%

*indicates that the 3 exclusions resulted in an n<6 and the exclusion of the entire data set for that TBBS.
ᵻ
Number of data points per model prior to exclusions.

Table 4
Maximum error (δDUTmax ; Eq. (8)) per DUT.

bias would be included in the constant, A. This would incorrectly
decrease δDUT,j if the bias was to increase the temperature measurement.
Conversely, a bias that decreased the temperature measurement would
incorrectly increase δDUT,j.
The test for outliers used in this study identified 43 data points, 41 of
which were excluded. Of the 41 exclusions, 38 (92%) were less than the
average of their data set, indicating that the outliers were biased to read
a lower temperature than the majority of readings. The purpose of
removing the outliers was to reduce the influence of bias when evalu­
ating the error. Removing the outliers decreased the number of DUTs
that did not meet the ASTM criterion of δDUTmax ≤ 0.3◦ C. However, when
the data were pooled from all ten DUTs of each model, 5 of the 6 models

97
S.J. Sullivan et al.
Table 5
Significant differences between NCIT models. Evaluation of whether different models reported a different output display temperature for the same input (TBBS)
temperature.
still did not meet the criterion. Model B had no outliers identified. In
contrast, Model D had the highest percentage of outlier exclusions, at
over 4%. Models A and F had ~ 2% of their readings excluded and the
exclusions for Models C and E were < 1%. Loss of infrared energy
detected by the device would result in a lower temperature read by the
device, therefore, these outliers may be an indicator of device usability
issues. For example, an NCIT that is not easy to grip and operate could
result in misalignment with the targeted measurement area.
When taking a subject’s forehead temperature, losses may occur if
external conditions are not controlled (e.g., excessive air movement), if
the distance to the target area is not compliant with the manufacturer’s
instructions, or if the surface is obscured (sweat, make-up or lotion on
the skin). These conditions can affect the NCIT measurement of the
forehead surface temperature and introduce error into the offset algo­
rithm that assumes clean dry skin of a given emissivity. For our labo­
ratory test, the design of the orifice of the horizontally-mounted conical
cavity blackbody source naturally isolates the NCIT sensor from unin­
tended radiation sources. The orifice of the blackbody is a single port,
such that when the face of the NCIT is placed against the port, the sensor
is exposed only to the blackbody radiation. Overall, many, but not all, of
the error sources in a typical use environment are mitigated or elimi­
nated by the use of the blackbody cavity.
No other previous studies were discovered to specifically evaluate
the performance of NCIT forehead thermometers against an ASTM
blackbody source. This is likely due to the limited interpretations that
can be made if the offset algorithm is unknown. ASTM E1965 provides a
test method for the evaluation of NCIT performance; however, this
method requires a controlled infrared radiation source and the adjust­
ment algorithm to be known. For 5 of the 6 models, an object mode is a
feature of the NCIT; however, it is intended for measuring objects or
fluids. In each case, the object mode has a wider accuracy specification
(± 0.5 to 1.0◦ C) for the same measurement range. In addition, the NCIT
may use a different emissivity setting from the adjusted mode that is
intended specifically for the forehead skin, making it inappropriate for
verification of the adjusted mode.
The findings of this study are consistent with past studies comparing
predictive thermometers to a reference thermometer. In a prior inves­
tigation, an evaluation of 3 fast read (contact) digital thermometers
98
Medical Engineering and Physics 93 (2021) 93–99
found the predictive algorithms deviated from a NIST calibrated refer­
ence by more than 1◦ C [9]. In another study by Tsai comparing infrared
ear thermometers, 4 of 14 DUTs did not meet the ASTM criterion when
evaluated using a blackbody source with a calibrated reference [10].
It would be expected that each unit of every model has the same
display output for the same TBBS, just as a temperature measurement
taken by two different NCITs of a single forehead would be expected to
be the same. However, the results showed the display temperatures to be
significantly different for different NCITs evaluated at the same TBBS
(Table 5) and that no model fully replicated the output of another. The
inter-model variability in the measured temperature could be attributed
a variety of sources: (a) usability or model-to-model variability
(Table 3), (b) differences between manufacturer’s algorithms and
emissivity settings, and (c) differences in the calibration set up used by
the manufacturers (for estimating the offsets) and those used in this
study.
5. Conclusions
This study elucidates the challenges of performing NCIT measure­
ment evaluations. The method used in this study allows for the evalu­
ation of NCIT error without knowing the manufacturer’s algorithm;
however, this method is cumbersome as it still requires the use of a
reference infrared radiation source (e.g., BBS). Five of the 6 models
evaluated had multiple outlier measurements identified and removed to
reduce the influence of measurement bias. Nonetheless, these same 5
models still did not meet the ASTM criterion, even after the outliers were
removed. Ideally, a clear means for the consumer to verify whether an
NCIT is working properly and providing an accurate measurement is
preferred. Model-to-model variability in displayed temperature is a
major source of concern and should be properly evaluated to ensure that
NCIT displayed temperatures are credible and consistent between
models. A clinical study using the same sets of DUTs will be conducted to
further evaluate their performance.
6. Disclaimer
The mention of commercial products, their sources, or their use in
S.J. Sullivan et al. Medical Engineering and Physics 93 (2021) 93–99

connection with material reported herein is not to be construed as either References

an actual or implied endorsement of such products by the Department of
Health and Human Services. [1] Fortuna EL, et al. Accuracy of non-contact infrared thermometer versus rectal
thermometry in young children evaluated in the emergency department for fever.
J Emerg Nurs 2010;36(2):101–4.
Funding [2] Franconi I, et al. Digital axillary and non-contact infrared thermometers for
children. Clin Nurs Res 2018;27(2):180–90.
[3] Wang K, et al. Non-contact infrared thermometers for measuring temperature in
None. children: primary care diagnostic technology update. The British journal of general
practice: the journal of the. R College General Practitioners 2014;64(627):e681–3.
Ethical approval [4] Ng DK, et al. Non-contact infrared thermometry temperature measurement for
screening fever in children. Ann Trop Paediatr 2005;25(4):267–75.
[5] Bitar D, Goubar A, Desenclos JC. International travels and fever screening during
Not required. epidemics: a literature review on the effectiveness and potential use of non-contact
infrared thermometers. Euro Surveill 2009;(6):14.
[6] Pompei F, Pompei M;. Non-invasive temporal artery thermometery: physics,
Declaration of Competing Interests
physiology and clincial accuracy. In medical thermometry for SARS detection, SPIE
defency and security symposium. 2004. Orlando, FL.
None declared; Funding: None; Ethical approval: Not required. [7] Vesnovsky O, et al. Modeling of differences between body core and forehead
temperatures measured by infrared thermometers. In: 2017 Design of Medical
Devices Conference; 2017.
Acknowledgements [8] ASTM. In: Standard Specification for Infrared Thermometers for Intermittent
Determination of Patient Temperature; 2016. E1965–98. 2016ASTM International
This project was supported through an FDA Medical Countermea­ West Conshohocken, PA.
[9] Vesnovsky O, et al. Performance testing of fast read digital thermometers. J Med
sures Initiative (MCMi) challenge grant and the Oak Ridge Institute for Devices-Trans Asme 2014;8(3):3.
Science and Education (ORISE) Research Participation Program at [10] Tsai SF. Comparison measurements of infrared ear thermometers against three
USFDA/CDRH. The authors thank FDA researchers Jon Casamento and types of blackbody sources. J Int J Thermophys 2010;31(8):1821–31.
Guha Suvajyoti for manuscript review.

99

View publication stats