Downloaded from https://iranpaper.

ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU
An Embedded Non-Contact Body Temperature Measurement
System with Automatic Face Tracking and Neural Network
Regression
1 2
Po-Wei Huang, Tzu-Hsuan Chang, Meng-Ju Lee , Tzu-Min Lin , Meng-Liang Chung, Bing-Fei Wu,
IEEE Fellow,
{abc1199281,qow110813, mlchung, bwu}@cssp.cn.nctu.edu.tw, faad3510@gmail.com ,
chateauxjolie@yahoo.com.tw

Abstract— In the last decade, many advances have been made
in the field of automatic temperature estimation, including
wearable sensor technologies (WST), infrared thermography
(IRT), and non-contact infrared thermometer (NCIT). In contrast
with the WST and IRT, NCIT is inexpensive without the risk of
potential skin irritation. Nevertheless, NCIT is limited in short
valid estimation distance (<12 cm), resulting in the non-
satisfaction of the surging application requirements nowadays.
This paper proposed an algorithm based on Neural Network
Regression not only to reduce the error from 0.6 ɗ to 0.12 ɗ ,
which is close to the medical instrument level, but as well to
lengthen the valid distance to the range between 50 cm and 100 cm.
Furthermore, this study developed an embedded automatic body
temperature estimation system which could continuously and
unconsciously measure the human temperature in real-time.
Integrated with face tracking and fuzzy-control of Pan-tilt unit, the
system ensures that human face is focused while measuring. With
wireless communication techniques, users can review their
physiological Information via App and Web, which is beneficial to
remote healthcare.
Key Words: Temperature Estimation, Neural Network Regression,
Automatic Face-tracking, Fuzzy-control.
I. INTRODUCTION
Automatic and continuous health status monitoring has
gathered great importance in the past few years. Among all the
indicators of health state in humans, body temperature is one
of the most significant indicators. Consequently, recent years
have seen increased attention being given to automatic
temperature monitoring technology. In comparison with
traditional techniques including digital axillary thermometer
(DAT), automatic body temperature thermometer can reduce
the labor cost and provide continuous, reliable and rapid
recording. Up to present, commonly used automatic
temperature evaluation techniques are wearable sensor
technologies (WST), infrared thermography (IRT), and non-
contact infrared thermometer (NCIT).
With the advance of technology, wearable devices have
integrated WST, wireless communication and low energy-
consumption microprocessor with high performance of data
processing. Nevertheless, WST may suffer from some

1
2
limitations. Long-term use of wearable devices might cause
skin irritation or allergic contact reactionsġ with unsuitable
choice of surface material [1].
In contrast, infrared (IR) based temperature evaluation is a
wear-free, non-invasive and safe technique [2]. Both IRT and
NCIT measure the radiation directly related to the skin
temperature (Tsk). In retrospect, IRT has been widely used
since the early 1960s in different areas [3]. The fact that IRT
for fever screening has been widely implemented by many
authorities for the mass screening purpose [4] is a typical
example.
Nonetheless, IRT is not without its flaws. Most of the IRT
systems are non-portable. Furthermore, the price of thermal
imaging systems remains high, making it difficult to promote.
In light of these concerns, NCIT serves as an alternative
solution. Daniel Santoso (2015) [5], for example, proposed an
inexpensive and portable system based on NCIT that screens
the influenza rapidly.
There is no qualified study without support of theories and
evidence. Since NCIT gathers the electromagnetic waves
released from the target, the law of optics must be mentioned
and highlighted. Chrzanowski (2001) [6] explored a review of
non-contact thermometry measurement, including the
geometry of source and receive, transmission of optical
radiation in atmosphere and so on. Likewise, Jonesġ(2002) [7]
has proposed the laws of thermodynamics since infrared
radiation is a type of heat energy as well. As for the emissivity,
which would influence IR based measurement due to different
surface material, has been indicated to be nearly constant at
0.98 for human and independent with color of skin [8] and
gender [9].
From the Biomedical viewpoint, classification of factors
that influence Tsk and the use of IR based thermometer has
been reported [10]. Although there might be several individual
factors, Teran (2012) [11] reported that the NCIT is strongly
correlated with rectal temperatures measured using mercury-
in-glass thermometers with difference less than 0.1 ć .
Similarly, Sollai (2016) [12] noted that NCIT is convincing in
comparison with DAT and infrared tympanic thermometers
Downloaded from https://iranpaper.ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU

(ITT). Chiappini (2011) [13] illustrated that NCIT is The proposed NNR system consists of four subsystems,
promising in children applications. comprising pan-tilt system, face tracking system, temperature
estimation model, and monitoring system. The following are
Nevertheless, the use of NCIT suffers from the limited details of these subsystems:
valid distance for biomedical applications, approximately 12
cm, depend on manufacturers. While considerable attention
has been paid in the past to research issues related to NCIT, A. Pan-tilt System
investigators on issues of lengthening the valid estimation The structure of the pan-tilt system can be divided into two
distance have emerged on slowly and in a scattered way. For parts, which is illustrated as Fig. 2. For the upper portion, there
the sake of various and increasing practical demand, NCIT are a tilt unit and a sensor platform, which bears the webcam,
estimation with relative long valid distance between 50 to 100 theġHC-SR04 module, and the TN901 module. On the other
cm is required. With the lengthened valid distance, there is no hand, the lower part carries the other motor that allows the pan
need for testers to adjust themselves to be measured. Instead, rotation of the upper structure.
the proposed system can automatically track human face and
record Tsk without users’ awareness.
In view of these concerns, the aim of this paper is therefore
twofold: (a) to enhance NCIT estimation performance with
novel temperature compensation technique. (b) to provide an
embedded automatic Tsk estimation system with real-time
face tracking technique.
Our results may help to clarify the influence of
environment temperature andġ distance between thermometer
and the target. The compensation methods reported here could
not only lengthen the valid NCIT estimation distance but as
well be beneficial to research attempting to increase the
accuracy and performance of NCIT. In addition, this paper
presents an embedded measuring system that could achieve the Figure 2. System Architecture
automatic, real-time performance with low-cost, improving
the quality of clinical care. B. Automatically Face Tracking System
Face tracking system is composed of two parts, face
The organization of the rest of the paper is as follows. tracking and motor control. For the former, Viola’s Robust
Section 2 begins with the total system architecture and deals Real-Time Face Detection [14] is implemented. Apart from
with the automatic face tracking system. Some basic that, we utilize an ROI optimization method to improve the
assumptions and detail statement of the temperature original ROI tracking system. As depicted in Fig. 3, integrated
compensation are highlighted in Section 3. In Section 4, with the smooth process and classical D (differential) control,
Neural Network Regression algorithm is depicted. Section 5 this system achieves real-time performance under QVGA
provides the experimental results and comparison. Section 6 resolution with Raspberry Pi 2.
concludes the observations and sets forth the future work.

II. SYSTEM ARCHITECTURE

As illustrated in the flow chart 1, the proposed system
captures images by the webcam to detect the human face and
keeps tracking on the target with step motors. As long as the
human face is focused, the raw data of temperature and
distance are recorded. Under the established mathematical
model, body temperature is evaluated; ultimately, the
temperature information is transmitted to the cloud server and
displayed on App or Web instantly.
Figure 3. Smooth Processing
In the motor control section (Fig. 4), this study singles out
the fuzzy theory to control step motors and facilitate our
system rotating more smoothly. Compared with traditional
control, fuzzy control performs more effectively on the motor
control issue.

Camera Center + E(t) Fuzzy V(t) Motor

- Controller

New Face Center Face

Detection

Figure 1 the NNR System Diagram Figure 4. Fuzzy Control


Downloaded from https://iranpaper.ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU

C. Temperature Estimation Model A. Linear Model without concerning the influence of Chest
ġġġġThis study proposed a temperature estimation algorithm According to the specifications of TN901 and the
based on Neural Network Regression (NNR). In an attempt to geometry of source/receiver [6], the received data represent
evaluate the body temperature, HC-SR04 ultrasound sensor the average of target temperature and ambient temperature
and TN901 infrared sensor have been utilized to evaluate the weighted by the area distribution ratio, as shown in Fig. 6.
distance within the range of 50 to 100 cm and the temperature
between 20 and 30 degrees centigrade. To verify the
effectiveness of temperature compensation procedure, a wide
variety of algorithms have been investigated along with
several experimental results.
D. Monitoring System
After the physiological information uploaded to the cloud
server via Bluetooth, a continuous body temperature report is
automatically produced for each patient as shown in Fig Ķ. Figure 6. Area relation between face and background

Supposing that the radiating energy is mainly delivered

from the face and given that is the area occupied by face,
we obtain

= × + × (1 − ) (2)

As shown in Fig. 7, the area occupied by face is inverse

square proportional to the distance. In line with the
specification, the Distance Scan Ratio (DSR) is 1. That is to
say, the diameter scanned by thermometers, which is defined
Figure 5. Temperature Record as R, equals to D multiplied by DSR.

All of the information can be looked over by either

smartphone App or website, making the remote health
management more automatic and uncomplicated.

III. PROBLEM FORMULATION

ġSuppose the relation between the body temperature and
infrared temperature observed from TN901 module can be
described as
= + (1)
Figure 7. Inverse square proportion
where
: the correct body temperature; Furthermore, it is assumed that the area occupied by
: the temperature observed by the TN901 module;
: the compensating temperature estimated by the face can be simulated as a circle with diameter f ,
proposed algorithm. Substituting DSR=1 and f into Eq. 2, we obtain
Given and , it is of ease to calculate the . So as
to determine , the distance between the target and the sensor,
§ § f ·2 · § f ·
2
which is defined as D, and the ambient temperature ( ) are ¨ ¨ ¸ uS ¸ ¨ ¸ uS
2 ¸ T u © 2 ¹
required. As a result, we have an estimation problem: Given To Ta u ¨1  © ¹ 2
D and , how to estimate accurately? ¨ §D· ¸ b § D ·2
Working with the estimation question requires ¨ ¨ ¸ uS ¸ ¨ ¸ uS
© ©2¹ ¹ ©2¹
investigation of factors that influence the evaluation. Since
thermometer records the radiating energy that is emitted from (3)
the source including background and human body, is
As the compensating temperature = − , Eq. 3
mainly influenced by [10]. The rest of this section depicts
can be written as follow:
two basic theories that deal with and estimate .
f2
Tc (Tb  Ta ) u (1  ) (4)
D2


Downloaded from https://iranpaper.ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU

The variance of the term Tb  Ta is relatively negligible

Among these 10 candidate orders in the range of {±1, ±2,
f2 ±3, ±4, ±5}, the result shows that the optimized order of Eq.
in regard with the term 1  . Consequently, the above
D2 8 is -1. Substituting this optimized order into Eq. 8, this result
equation may be reduced to Eq.5. With adequate collected argues that (D) and distance are in direct proportion. As
data, the coefficients, D lm1 and E lm1 , can be optimized by illustrated in Fig 9, the area occupied by body is also
proportional to the distance; therefore, the direct proportion
Least Square Error (LSE) and thus the first candidate relationship between D and (D) is reasonable.
evaluation algorithm has been derived as Eq. 5.

Tc D lm1  Elm1 u D 2 (5)

B. Linear Model concerning the influence of Chest

In the deriving process of Eq. 5, it is supposed that the
radiating energy produced by the body other than face may be Figure 9 proportional relation to Distance
negligible. Nevertheless, in pursuing the superior estimation
performance, experimental results argue that radiating energy Identically, the coefficients, Dlm2 and Elm2 , are
generated by the body, mainly the chest, should be taken into
account. In accordance with this consideration, we regard the determined by LSE, and the new estimation algorithms are
area occupied by face or chest as a function of derived as follow.
distance, (D). Hence, Eq. 2 should be rewritten as new
estimation equation. = × + (9)

§ · IV. NEURAL NETWORK REGRESSION

¨ ¸
To
A ( D)
Ta u ¨1  all 2 ¸  T u Aall  D  (6) ġġġBoth the above-mentioned algorithms consider the term
¨ §D· ¸ b § D ·2 Tb  Ta negligible. Nevertheless, there is likelihood that the
¨ ¨ ¸ uS ¸ ¨ ¸ uS
© ©2¹ ¹ ©2¹ influence caused by Ta is noticeable. For the sake of this issue,
we proposed a neural network regression (NNR) model. NNR
Likewise, we substitute = − into Eq. 6, is a Multi-layer perceptron network and trained by back-
deriving the new equation. propagation with Levenberg-Marquardt algorithm, which is
§ · commonly used for least squares curve fitting problem.
¨ ¸
A 
Tc (Tb  Ta ) u ¨1  all 2
D  ¸ (7) Several implementation issues are listed below:
¨ §D· ¸
¨ ¨ ¸ uS ¸ A. Net Structure
© ©2¹ ¹
Following are descriptions of the NNR structure. There
Similar to the derivation mentioned in Eq. 5, the term
are two neurons in the input layer with tangent sigmoid as the
Tb  Ta is negligible. We will now reduce Eq. 7 to a simpler transfer function. The number of hidden layers is six with
form. tangent sigmoid transfer function and eight neurons in each
= × + (8) layer. The activation function used in the output layer is pure
linear function with one output neuron.
To determine the order of Eq. 8, we utilize LSE and
B. Parameters
Variable Selection with 10-fold Cross Validation, as depicted
in Fig. 8. NNR is trained with the following fixed parameter
settings. The learning rate is 0.05, with maximum 4000
epochs. The performance goal is 0.0001 with minimum
gradient rate g m 10 20 . Other coefficients of the adaptive
parameter P are summarized as follow:
1) Initial value: 0.001
2) Decrease step: 0.1
3) Increase step: 10
10
4) Maximum value: 10
C. Data Preprocessing
Before training, regularization of input data and the
Figure 8 Model Selection with 10-fold Cross-validation target has been proven frequently effective. Consequently, we


Downloaded from https://iranpaper.ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU

linearly regularize the input and output data into the range
between -1 and 1.
With the implement issues mentioned above, we have
expounded the details of NNR proposed in this paper.

V. EXPERIMENTAL RESULTS AND DISCUSSION

A. Purpose of the Experiment

So as to estimate the proposed compensation model, the
study designs the following experiment to collect the required
data comprising observed temperature, ambient temperature,
and distance.
B. Design of the Experiment
First, with fixed background and distance between the
instrument and the face, the system measures one group of data
each ten centimeters from 50 to 100 cm. To avoid the influence Figure 10 NNR response.
of distance outliers, only the data whose distance is limited in
the range of ±3 cm from standard be taken into account. Each Though the performance is mainly determined by the
experiment group contains consecutive 250 data. In order to criteria mentioned-above. The response of NNR is also
truly reflect the potential time variant influence, the study commonly used. The result of the regression analysis between
measures at different time to collect different environmental the response of NNR and the target, tympanic temperature is
temperatures. illustrated as Fig. 11. As the R-value of the regression
between the NNR response and target is 0.9953, very close to
C. Data preprocessing
1, it is convincing that NNR is a nice fit.
This study is strict to the received data. There are at least
250 data recorded in each experiment group and the process
unit of the data preprocessing is the experiment group.
There are two steps in the data preprocessing. First, the
data whose difference from group average is more than 0.5 ℃
are regarded as outliers and removed. Second, investigating
the standard deviation of each group is essential. Since the
large standard deviation means the ambient temperature is
floating and the recorded results are non-convincing, the
groups whose standard deviation is more than 0.19 ℃ are
abandoned. After the data preprocessing, we have obtained the
data suitable for further analysis.
D. Database
With the approaches mentioned above, there are totally
14533 valid data for analysis. Each datum consists of infrared
temperature, environmental temperature, distance, and Figure 11 Regression of NNR response and the target.
tympanic temperature to verify the accuracy of Tsk
estimation. G. Baseline
E. Performance criteria In addition to the Eq. 5 and Eq. 9, for the purpose of
figuring out the optimized estimation algorithm, several
The performance criteria of each algorithm is defined as potential algorithms are also derived in this study including
average of error and standard deviation of error. The Support Vector Machine (SVM) regression, linear regression
definition of error is the absolute difference of the calibrated (LR), and Regression Tree (RT).
temperature and the tympanic temperature. There are many types of potential LR models including
F. NNR result correlated component regression (CCR), robust linear
regression (RLR), and generalized linear regression (GLR).
The response of NNR is depicted as Fig. 10, with the mean
All of the regression models mentioned above have been
error: 0.1155 ℃ , and the standard deviation of the error:
surveyed and implemented.
0.1119℃. Each group of blue nodes represents experiment
The followings are several implementation issues of RT:
group containing approximate 250 data. In contrast with the
The definition of node impurity is the mean-square error. As
original mean error (0.6 ℃ ) based on the specification of
for the depth controller, several minimum of parent node size
TN901, NNR serves as an excellent estimation algorithm.
(PNS) has been used including 1000, 600 and 300. The
specific value 300 comes from the design of experiment. In


Downloaded from https://iranpaper.ir
https://www.tarjomano.com/order
3URFHHGLQJVRI,QWHUQDWLRQDO$XWRPDWLF&RQWURO&RQIHUHQFH&$&6
7KH(YHUJUHHQ/DXUHO+RWHO7DLFKXQJ7DLZDQ1RYHPEHU
this study, around 250 data are collected for every specific
pairs of D and Ta . If the size of PNS is smaller or around 250,
it leads to the exceedingly possibility that the trained model is
over fitted. Hence, the minimum constraint of PNS is set as
300.
So far, we have elaborated the fundamental settings for
comparison with NNR.
H. Comparison
If there is no compensation process, the mean error of
TN901 is 0.6 ℃ . The performance of all the mentioned-above
algorithms are shown as Fig.12.
Figure 12 Algorithms Comparison
Figure. 12 illustrates that NNR is actually the most
effective fitting model, which minimizes the measuring error
and the standard deviation to smaller than 0.12℃. The RT
algorithm ensues with slight rise from NNR by roughly 0.02
℃ . Lastly comes the SVM, which reduces just 0.07℃ in
contrast with the original error.
On the other hand, the results indicate that the algorithm
based on Eq. 5 is superior to the algorithm based on Eq. 9,
which elaborates that the radiation released from chest indeed
influences the estimation results.
VI. CONCLUSION
In this paper, we have proposed a novel NCIT
compensation algorithm based on NNR which lengthens the
valid estimation distance from original distance (<12 cm) to
the range between 50 cm and 100 cm. The proposed algorithm
in the meantime effectively reduces the error from 0.6 ℃ to
smaller than 0.12 ℃ .
Utilizing the proposed algorithm, this study has as well
implemented an automatic body temperature estimation
system. Integrated with real-time face tracking techniques,
fuzzy control of pan-tilt subsystem, cloud server techniques,
and the embedded subsystem, this study has achieve the long-
term, automatic, and inexpensive requirements of temperature
estimation.
Some questions will be explored in the future. As the
limitations of experiment environment, the emissivity of the
background is constant and the range of the ambient
temperature are limited (20-30 ℃). The problems related to
this still open widely and are among our future research goals.

ACKNOWLEDGMENT
This work was supported by MOST 104-2221-E-009-190-
REFERENCES
[1] Z. Zhu, T. Liu, G. Li, T. Li and Y. Inoue, "Wearable Sensor Systems
for Infants", Sensors, vol. 15, no. 2, pp. 3721-3749, 2015.
[2] F. Ring, "Pioneering progress in infrared imaging in medicine",
Quantitative Infrared Thermography Journal, vol. 11, no. 1, pp. 57-
65, 2014.
[3] B. Brattstrom, "A Preliminary Review of the Thermal Requirements
of Amphibians", Ecology, vol. 44, no. 2, pp. 238-255, 1963.
[4] Mercer, James B., and E. Francis J. Ring. "Fever screening and
infrared thermal imaging: concerns and guidelines.", Thermology
International, vol. 19, no. 3, pp. 67-69, 2009.
[5] Santoso, Daniel, and F. Dalu Setiaji., "Non-contact portable infrared
thermometer for rapid influenza screening.", International Conference
on Automation, Cognitive Science, Optics, Micro Electro-Mechanical
System, and Information Technology (ICACOMIT). IEEE, 2015.
[6] K. Chrzanowski, Non-contact thermometry. Warsaw: SPIE Polish
Chapter, 2001. Chrzanowski, K. (2001).
[7] B. Jones and P. Plassmann, "Digital infrared thermal imaging of
human skin", IEEE Eng. Med. Biol. Mag., vol. 21, no. 6, pp. 41-48,
2002.
[8] J. Steketee, "Spectral emissivity of skin and pericardium", Physics in
Medicine and Biology, vol. 18, no. 5, pp. 686-694, 1973.
[9] J. Christensen, M. Vaeth and A. Wenzel, "Thermographic imaging of
facial skin—gender differences and temperature changes over time in
healthy subjects", Dentomaxillofacial Radiology, vol. 41, no. 8, pp.
662-667, 2012.
[10] I. Fernández-Cuevas, J. Bouzas Marins, J. Arnáiz Lastras, P. Gómez
Carmona, S. Piñonosa Cano, M. García-Concepción and M. Sillero-
Quintana, "Classification of factors influencing the use of infrared
thermography in humans: A review", Infrared Physics & Technology,
vol. 71, pp. 28-55, 2015.
[11] C. Teran, J. Torrez-Llanos, T. Teran-Miranda, C. Balderrama, N. Shah
and P. Villarroel, "Clinical accuracy of a non-contact infrared skin
thermometer in paediatric practice", Child: Care, Health and
Development, vol. 38, no. 4, pp. 471-476, 2011.
[12] S. Sollai, C. Dani, E. Berti, C. Fancelli, L. Galli, M. de Martino and E.
Chiappini, "Performance of a non-contact infrared thermometer in
healthy newborns", BMJ Open, vol. 6, no. 3, p. e008695, 2016.
[13] E. Chiappini, S. Sollai, R. Longhi, L. Morandini, A. Laghi, C. Osio,
M. Persiani, S. Lonati, R. Picchi, F. Bonsignori, F. Mannelli, L. Galli
and M. de Martino, "Performance of non-contact infrared thermometer
for detecting febrile children in hospital and ambulatory settings",
Journal of Clinical Nursing, vol. 20, no. 9-10, pp. 1311-1318, 2011.
[14] P. Viola and M. Jones, "Robust Real-Time Face Detection",
International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154,
2004.