You are on page 1of 8

Open Physics 2020; 18: 566–573

Research Article

Gangqin Huang, Zhangfa Yan, Tiantian Dai*, Ren-Guey Lee, and Qingyang Wei*

Simultaneous measurement of ionizing


radiation and heart rate using a smartphone
camera
https://doi.org/10.1515/phys-2020-0181 measure HR simultaneously with smartphone camera
received July 14, 2020; accepted August 12, 2020 lens covered by a finger. We have been working on the
Abstract: Complementary metal oxide semiconductor development of an Android phone application based on
(CMOS) cameras have been used to detect ionizing the algorithms.
radiation when shaded from light or under the condition Keywords: complementary metal oxide semiconductor
of static scenes. We propose a novel idea for simulta- sensors, heart rate measurement, ionizing radiation
neous measurement of ionizing radiation and heart rate detection, smartphone
(HR) by using a smartphone with its CMOS camera lens
covered by a finger in one measurement. Verified
experiments were conducted: videos were recorded
using a Xiaomi8 smartphone when the camera lens 1 Introduction
was covered by a finger and a light-tight tape for
controlled experiment, with a ∼33 mCi 99mTc liquid The extensive application of nuclear technology in
radioactive source at six different distances (from 20 to industry, agriculture, medical science, and other fields
70 cm, step size 10 cm) from the camera. The exact HRs has greatly promoted economic construction and devel-
were measured using an oximeter at the same time. opment of the world. However, ionizing radiation can
Image processing algorithm is proposed to extract cause damage to humans and animals, such as genetic
radiation events and HR in the same videos. Results mutations and chromosomal aberrations, which cause
show that: (1) frame numbers containing radiation- one or even several generations of damage and even lead
related bright spots in two experiments have a linear to death [1]. Potential safety hazards that nuclear
relationship, and they are inversely proportional to the technology may bring cannot be ignored, and various
square of the distance between the camera and radiation radioactivity-related accidents occurred, such as nuclear
source; (2) the HR difference between the video process- leakage of Fukushima Daiichi nuclear power plant in
ing and the oximeter result is less than 2 beats per March 2011 [2] and Jilin Daqing Drilling and Detecting
minute. In conclusion, it verifies that the proposed Well Company lost the container containing radioisotope
method is available to detect ionizing radiation and in September 2016 [3]. The Chernobyl accident in 1986
caused many severe radiation effects, and of the 600
workers present at the site, 134 received high doses
 (0.8–16 Gy) and suffered from radiation sickness, of
* Corresponding author: Tiantian Dai, Department of Radiation whom, 28 died in the first 3 months [4]. Therefore,
Oncology, China-Japan Friendship Hospital, Beijing, 100029, China, nuclear radiation detection is crucial to ensure nuclear
e-mail: maxinedtt@163.com
security. It can reduce the probability of a nuclear
* Corresponding author: Qingyang Wei, Beijing Engineering Research
Center of Industrial Spectrum Imaging, School of Automation and accident on one hand and reduce its damage after a
Electrical Engineering, University of Science and Technology Beijing, nuclear accident on the other.
Beijing, 100083, China, e-mail: weiqy@ustb.edu.cn Traditional technologies use dedicated radiation
Gangqin Huang, Zhangfa Yan: Beijing Engineering Research Center detection instruments such as gas detectors, scintillator
of Industrial Spectrum Imaging, School of Automation and Electrical
detectors, and semiconductor detectors [5,6]. Nowadays,
Engineering, University of Science and Technology Beijing, Beijing,
100083, China
various kinds of novel radiation source locating or
Ren-Guey Lee: Department of Electronic Engineering, National imaging systems [7–10] have been developed for radia-
Taipei University of Technology, Taipei, 106, Taiwan tion monitoring and fast locating the radiation sources.

Open Access. © 2020 Gangqin Huang et al., published by De Gruyter. This work is licensed under the Creative Commons Attribution 4.0
International License.
Simultaneous measurement of ionizing radiation and HR using a smartphone camera  567

However, these traditional detectors and novel imaging 2 Materials and methods
systems are expensive and require a lot of post-
maintenance, but the relatively low probability of
2.1 Experiments
nuclear-related accidents/incidents makes the use of
these detectors and systems on a large scale cost-
effective. A certain smartphone (Xiaomi8) with dual 12-megapixel
In recent years, complementary metal oxide semi- rear camera (Sony IMX363) was used for studies. The
conductor (CMOS) cameras have been used for X/γ-ray main rear camera was utilized to record videos. The
detection and classification [11–14]. This promising image sensor of the camera has a size of 1/2.55 inches
direction has advantages of higher penetration rate and and a single pixel-type of 1.4 µm. The video frame rate
lower cost compared to traditional methods. Studies recorded by the smartphone is 30 fps, and the image size
have shown that random bright spots are generated in is 1,920 × 1,080. The radioactive source used in
images recorded by CMOS image sensors under irradia- experiments is a ∼33 mCi 99mTc liquid radioactive source
tion of X/γ rays [11–13]. Xu et al. studied the potential of (2 mL) with the γ-ray energy of 140 keV and the decay
CMOS image sensors and found that both the mean pixel half-life of 6.02 h. Two groups of experiments were
value and the cumulative pixel value have a linear carried out. All experiments were carried out in a room
relationship with radiation dose rate [14–16]. Our group with uniform and constant artificial illumination. In
successfully extracted radioactive events in videos and group 1, the smartphone camera lens was completely
found that the frequency of the bright spots is positively covered by a finger of an adult male subject, and videos
correlated with the radiation intensity [17,18]. Several were recorded when the smartphone was set at six
algorithms were developed to identify the radioactive different distances from the radioactive source to obtain
events in images or videos recorded by CMOS cameras different radiation intensities, ranging from 20 to 70 cm
[18–20]. Since smartphones have powerful abilities for with a step size of 10 cm. At the same time, we measured
information collection and computational analysis and the HR for each experiment using a Heal Force Prince-
contain CMOS cameras, using a smartphone to detect 100F Oximeter [28]. Figure 1 shows the details of
ionizing radiation has become a research hotspot and measurement in group 1. In group 2, the smartphone
many smartphone applications (application [APP]) have camera lens was completely covered by a light-tight
been developed and released. Quick and Drukier et al. tape. For controlled experiments, six sets of videos were
released ionizing radiation detection APPs WikiSensor recorded under the same conditions as group 1. Each
[19] and GammaPix [20], respectively, in 2011. However, video of the two group experiments was recorded for
both of the APPs only can be used under the condition 1 min continuously. The data were processed by using
that the smartphone camera lens is completely shielded MATLAB R2016b (MathWorks Inc., Natick, MA, USA).
from light by a light-tight device. A real-time ionizing
radiation detection APP called RayDose is developed by
our group in 2017 [18], which can be used in a static and Smartphone camera lens
Probe
non-shading condition. Our group is further working on Heal Force Prince -
the research of nuclear radiation detection based on 100F Oximeter

uncovered surveillance CMOS camera under dynamic


scene [21].
In this paper, we propose a novel idea for achieving
ionizing radiation detection by using a smartphone with
the CMOS camera lens covered by a finger instead of a
light-tight device, which can also measure the heart rate
(HR) simultaneously. The algorithm for HR measurement
is based on the non-invasive photoplethysmography
(PPG) technology, which was first proposed by Hertzmen
et al. in the 1930s [22]. Many smartphone APPs for HR
monitoring are based on this principle [23–27]. Algo-
rithms were developed to extract the radiation events
and HR simultaneously. Verified experiments were
conducted in the following section. Figure 1: The details of measurement in group 1.
568  Gangqin Huang et al.

2.2 Image processing

The first procedure is to obtain radiation events in finger


videos. In this procedure, we combine the modified
inter-frame difference method with a threshold method
to identify if a specified frame contains radiation-related
bright spots and count the number of target frames in
1 min. The second procedure is to extract the PPG signals
in the same finger videos and calculate the HRs.
Figure 2 shows the frame image with a bright spot in
the bottom left corner recorded by the smartphone with
the finger-covered camera lens. Figure 3: A superimposing image with all bright spots of a 1 min
Figure 3 is a superimposing image with all bright video recorded by the smartphone with the finger-covered camera
spots of a 1 min video recorded by the smartphone with lens using the 99mTc source placed at a distance of 20 cm from the
smartphone.
the finger-covered camera lens and the 99mTc source
placed at a distance of 20 cm from the smartphone. It can
be observed that positions where the bright spots appear image is calculated by a combination of forward inter-
in the image are random. Under the irradiation of γ-rays, frame difference method and post inter-frame difference
bright spots are also generated and can be clearly method, which can improve the accuracy. Assume that
distinguished in the images when smartphone camera the grayscale images of three consecutive frames in a
lens was covered by a finger. video are In − 1, In , and In + 1, where In is the current frame.
Besides, based on our previous research results [14], Then the forward difference image D1 and the backward
bright spots generated by ionizing radiations are difference image D2 are calculated as follows:
almost impossible to appear at the same position of
 D1 = In − In − 1
adjacent frame images. An inter-frame difference  (1)
method combined with a threshold method can be  D2 = In − In + 1.
used to extract radiation-related bright spots in videos. Then the current difference image is calculated as
The algorithm has the advantages of simple implemen- follows:
tation, low programming complexity, insensitivity to
Vn = min(D1 , D2 ). (2)
scene light changes, and strong adaptability to dynamic
environment. Details of the algorithm are described Figure 4 shows the grayscale distributions in two
below: typical images Vn , Figure 4(a) is the grayscale distribution
Step 1. The modified inter-frame difference method of Vn for current frame with bright spots, and Figure 4(b)
calculates the inter-frame difference image Vn . Since the is the grayscale distribution of Vn for current frame
image background is changed due to blood flow changes without bright spots. We can note that Figure 4(a) has
in the finger, in this work, the inter-frame difference pixels with large gray value.
Step 2. The threshold method calculates the max-
imum value η of Vn and compares η with the threshold T.
If η > T, the current frame will be judged as a frame
containing bright spots. The setting of the threshold T
affects the detection sensitivity: higher threshold will
decrease the detection sensitivity, and lower threshold
will cause a false detection. The noise approximately
obeys a normal distribution, therefore, the probability of
mistaking noise beyond +4 sigma is 0.00158%. Thus, Tn
can be calculated by equation (3).
Tn = μn + 4σn, (3)

where μn is the mean value of η{1 − n} and σn is the


Figure 2: A frame with a bright spot of the video recorded by the standard deviation of η{1 − n}, which are dynamically
smartphone with the finger-covered camera lens. calculated by equation (4).
Simultaneous measurement of ionizing radiation and HR using a smartphone camera  569

Figure 4: The grayscale distributions of two typical images Vn and (a) the grayscale distribution of Vn for current frame with bright spots and
(b) is the grayscale distribution of Vn for current frame without bright spots.

 n−N−1 1 variance. In this work, we used the MATLAB PCA


 μn = μn − 1 n − N + ηn n − N function to carry out PCA. HR measurement algorithms
 , (4)
 σn = σn − 1 2 n − N − 1 + (η − μ )2 1 are described below:
 n n
 n−N n−N Step 1. Decompose video frames into red, green, and
blue (RGB) channels and get the R, G, B PPG signals by
where N is the number of frames containing bright spots
taking the mean pixel values of each frame from R, G, B
in the first n frames N. If ηn > Tn − 1, the current frame
channels.
contains bright spots, ηn will not be used to compute μn
Step 2. Conduct PCA, the input vector is R, G PPG
and σn .
signal combination as Lamonaca’s research has proved
The algorithms were used to get the number of
the R channel signal can best reflect heart-beat
frames N1 containing bright spots in 1 min finger videos
when the smartphone camera was placed at the distance information [25]. Besides, the absorption of yellow and
ranging between 20 and 70 cm from the 99mTc source and green light by the blood is higher. Figure 5 shows the R,
the number of frames N2 containing bright spots in 1 min G PPG signals for 15 s.
videos of group 2. Since the radiation intensity is Figure 6 is the result of PCA. Principal component
inversely proportional to the distance from the center that contains information about HR was selected as the
of the radiation source. If the number N1 is proportional first principle component.
to the square of the distance between camera lens and Step 3. Calculate HR. The PPG signal is a periodic
radiation source, the number N1 is proportional to the pulse, and its period is equal to the cardiac cycle period.
radiation intensity. Besides, based on our previous We can detect the peaks of the first principle component
research [17,18], the number N1 and N2 should have a signal and get the time interval DT (unit: second)
linear relationship. between adjacent peaks. The HR can be calculated by
The algorithm for HR measurement is based on the equation (5):
PPG technology [22–27]. Principal component analysis 60
(PCA) is used for data analysis. The PCA is a technique HR = . (5)
DT
commonly used for data reduction in statistical pattern
recognition and signal processing [29]. The basic The HR is between 42 and 180 bpm (which is equal
principle of PCA is to find an orthogonal transformation to 0.7–3.0 Hz), in order to avoid false detection of the
matrix U to convert vector X = [X1, X2,…, XN ]T into a venous wave peaks and the gradual wave peaks in the
vector Y = [Y1, Y2,…, YN ]T , the transformation is Y = UX , signal as the peaks of the PPG signal, we limited the time
then Y1 is the first component of X, Y2 is the second difference between the peaks by 0.5 s (the median of the
component of X, and so forth. The columns of matrix U HR range is about 120 bpm). We will detect the peaks in
are the principle components. They are the eigenvectors 1 min and calculate the average value of DT to get the
of the covariance matrix, ordered with decreasing average HR in 1 min.
570  Gangqin Huang et al.

Figure 5: The R, G PPG signals for 15 s.

3 Results and discussion (Figure 7a and b) and 50 cm (Figure 7c and d) from the
99m
Tc source. It can be seen from the pictures that under
the condition that the smartphone camera lens comple-
3.1 Results of ionizing radiation detection
tely covered by a finger or a light-tight tape, the value of
η is randomly appearing at a large value. Meanwhile, the
Videos recorded by the smartphone with its camera lens number of η at large value in 20 cm is more than in
completely covered by a light-tight tape and a finger 50 cm, which means the number of η at large value in
were processed by our proposed algorithm. Figure 7 high radiation intensity is more than that in low
shows the distribution of η for the video recorded with radiation intensity.
camera covered by a light-tight tape (Figure 7a and c) The relationship between the number N and the
and a finger (Figure 7b and d) for 1,800 frames (equal to distance D is shown in Figure 8. The blue curve fitting is
9.4769 × 104
1 min) when the camera was placed at distance of 20 cm N1 = D2
for finger video, the red curve fitting is

Figure 6: Result of PCA of RG signal combination.


Simultaneous measurement of ionizing radiation and HR using a smartphone camera  571

Figure 7: Distribution of η for 1,800 frames with smartphone cameras covered by a light-tight tape (a and c) and a finger (b and d) when the
camera was placed at distance of 20 cm (a and b) and 50 cm (c and d) from the 99mTc source.

1.2047 × 105 not suitable for radiation detection asking for high
N2 = D2
for light-tight video. Thus, both N1 and N2
are approximately inversely proportional to the square of accuracy. Besides, since different types of smartphones
distance, which means the number N is proportional to have various CMOS cameras, the detection efficiency will
the radiation intensity. As the distance D from the be different. However, it has advantages of low-cost,
camera lens to radioactive source is not absolutely easy access to everyone. The algorithm is also simple
accurate, there is a large error in the fitting result and can be used for real-time detection. The accuracy of
at 20 cm. the proposed method is slightly lower than the camera
Figure 9 shows the relationship between N1 and N2
for distances ranging from 20 to 70 cm. It shows that N1
and N2 have a linear relationship, and the linear fitting
result is N1 = 0.9576N2 − 14.6 with R2 = 0.978. N1 has
lower count than N2 , and the ratio of N1 and N2 is
0.9576:1. There are two main reasons for this phenom-
enon: (1) the finger attenuates gamma-rays, the number
of bright spots generated in image is reduced; (2) since
the finger cannot completely block the light, the noise of
finger video image is higher than the light-tight tape
covered video image.
Therefore, under the condition that the smartphone
camera lens is completely covered by a finger, the
camera can record the video to realize the ionizing
radiation detection. However, since the CMOS camera is Figure 8: The relationship between the number N and the distance
not sensitive than the dedicated detectors, the method is from smartphone to radioactive source.
572  Gangqin Huang et al.

Table 1: Results of the HR measurement

HR (bpm) Distance (cm)

20 30 40 50 60 70

Our measurement 77.5 75.9 74.4 75.1 76.3 74.6


Oximeter measurement 77.8 76.2 75.1 75.9 74.5 74.9
The difference −0.3 −0.3 −0.7 −0.8 1.8 −0.3

detection. Meanwhile, we successfully calculated the HR


from the finger videos, and the HR difference between
our measurement and the oximeter measurement is less
Figure 9: The relationship between the number N1 and N2. than 2 bpm. In conclusion, the proposed method can
realize ionizing radiation monitoring and HR detection
lens completely blocked. However, the method is very simultaneously by using a smartphone camera.
simple and convenient, and it can be used for fast We have been working on the development of
detection everywhere without smartphone camera lens an APP based on the algorithm to realize detecting
covered by a light-tight tape but a finger. ionizing radiation and human HR simultaneously. As
smartphones are used by almost everyone everywhere, it
is convenient for the general public to check whether
there are radiation surroundings and measure HR
3.2 Results of HR measurement anytime.

We extracted PPG signals from six sets of videos in group Acknowledgments: This research was supported by the
1 and detected the peaks in 1 min to calculate the National Natural Science Foundation of China (grant
average HR of 1 min. The results of our measurement and number 11975044), Fundamental Research Funds for the
the oximeter measurement are listed in Table 1. The Central Universities (grant number FRF-TP-19-019A3),
biggest difference between our measurement and oxi- Joint Research Project between University of Science
meter measurement is 1.8 bpm, less than 2 bpm. It and Technology Beijing and National Taipei University
verifies that the proposed method can measure HR of Technology (grant number TW201708), and Science &
with a good accuracy. Technology on Reliability & Environmental Engineering
Laboratory (grant number 6142004180205).

4 Conclusions
References
In this paper, we proposed a method for radiation
detection using a smartphone camera by partly shading [1] UNEP. Radiation: effects and sources, 1st edn. New York, USA:
the sensor from light with a finger and measure HR United Nations; 2016. 11–24.
simultaneously. With the proposed algorithm, we iden- [2] Hirose K. Fukushima Daiichi nuclear power plant accident:
tified whether the recorded video contained the radia- summary of regional radioactive deposition monitoring
results. J Environ Radioact. 2011;2012(111):13–17.
tion-related spots. The result shows that the number of
[3] Sohu, China [cited 2020 Jan 10]. Available from: http://www.
frames with bright spots in 1 min inversely proportional sohu.com/a/114982338_116897.
to the square of the distance from smartphone camera to [4] UNSCER. The Chernobyl accident [cited 2020 Feb 5]. Available
radioactive resource. It means it has a positive correla- from: http://www.unscear.org/unscear/en/chernobyl.html.
tion with the radiation intensity of the sensor exposure [5] Baek CH, Lee SJ, Choi Y, Chung YH. Optimization of large-angle
to. In addition, the number of frames when the camera pinhole collimator for environmental monitoring system. IEEE
Trans Nucl Sci. 2010;57(3):1404–8.
was covered by a finger and the number when the
[6] Bolotnikov AE, Camarda GS, Cui Y, Yang G, Hossain A, Kim K,
camera was covered by a light-tight tape have a linear et al. Characterization and evaluation of extended defects in
relationship. It verifies that the method is available for CZT crystals for gamma-ray detectors. J Cryst Growth.
the camera with a finger pressed on to realize radiation 2013;379:46–56.
Simultaneous measurement of ionizing radiation and HR using a smartphone camera  573

[7] Wei Q, Yang Z, Dai T, Novel A. Radiation source location https://newatlas.com/wikisense-radiation-detector-app/


detector based on attenuation. In: 2018 IEEE Nuclear Science 20294/
Symposium and Medical Imaging Conference Proceedings [20] Drukier GA, Rubenstein EP, Solomon PR, Serio MA,
(NSS/MIC), 2018 November 10–17. Sydney, Australia: IEEE; Wójtowicz MA. Low cost, pervasive detection of radiation
2018. p. 1–3. threats. In: Proceedings of 2011 IEEE International
[8] Ye Q, Fan P, Wang R. A high sensitivity 4π view gamma imager Conference on Technologies for Homeland Security (HST),
with a monolithic 3D position-sensitive detector. Nucl Instrum 2011 November 15–17. Waltham, MA, USA: IEEE; 2011.
Methods Phys Res A. 2019;937:31–40. p. 365–71.
[9] Sun S, Zhang Z, Shuai L. Development of a panorama coded- [21] Yan. Z, Wei Q, Huang G, Hu Y, Zhang Z, Dai T. Nuclear radiation
aperture gamma camera for radiation detection. Radiat Meas. detection based on uncovered CMOS camera under dynamic
2015;77:34–40. scene. Nucl Instrum Methods Phys Res A. 2020;956:163383.
[10] Zhang J, Li C, Pang X. Development of a 3-D scintillator [22] Hertzman AB. The blood supply of various skin areas as
detector for compton imaging based on laser engraving. IEEE estimated by the photoelectric plethysmograph. Am J Physiol.
Trans Nucl Sci. 2020;67(7):1691–8. 1938;124(2):328–40.
[11] Cheng Q, Ma C, Yuan Y, Wang F, Jin F, Liu X. X-ray detection [23] Grimaldi D, Kurylyak Y, Lamonaca F, Nastro A.
based on complementary metal-oxide-semiconductor sen- Photoplethysmography detection by smartphone’s video
sors. Nucl Sci Tech. 2019;30:43–8. camera. In: Proceedings of the 6th IEEE International
[12] Magalotti D, Bissi L, Conti E. Performance of CMOS imager as Conference on Intelligent Data Acquisition and Advanced
sensing element for a real-time active pixel dosimeter for Computing Systems, 2011 September 15–17. Prague, Czech
interventional radiology procedures. J Instrum. Republic: IEEE; 2011. p. 488–91.
2014;9(1):C01036. [24] Scully CG, Lee J, Meyer J, Gorbach AM, Granquist-Fraser D,
[13] Lane DW. X-ray imaging and spectroscopy using low cost Mendelson H, et al. Physiological parameter monitoring from
COTS CMOS sensors. Nucl Instrum Methods Phys Res A. optical recordings with a mobile phone. IEEE Trans Biomed
2012;284:29–32. Eng. 2012;59(2):303–6.
[14] Xu S, Zou S, Huang Y. γ -ray detection using commercial off- [25] Lamonaca F, Kurylyak Y, Grimaldi D, Spagnuolo V. Reliable
the-shelf CMOS and CCD image sensors. IEEE Sens J. pulse rate evaluation by smartphone. In: 2012 IEEE
2017;17(20):6599–604. International Symposium on Medical Measurements and
[15] Xu S, Zou S, Han Y, Qu Y, Zhang T. Video monitoring Applications Proceedings, 2012 May 18–19. Budapest,
application of CMOS 4T-PPD-APS under γ-ray radiation. Hungary: IEEE; 2012. p. 1–4.
Sensors. 2019;19(2):359. [26] Nam YY, Kong Y, Reyes B, Reljin N, Chon KH. Monitoring of
[16] Xu S, Velthuis J, Wu Q, Han Y, Lin K, Beck L, et al. Effect of heart and breathing rates using dual cameras on a smart-
commercial off-the-shelf MAPS on γ-ray ionizing radiation phone. PLoS One. 2016;11(3):e0151013.
response to different integration times and gains. Sensors. [27] Banitsas K, Pelegris P, Orbach T, Cavouras D, Sidiropoulos K,
2019;19(22):4950. Kostopoulos S. A simple algorithm to monitor HR for real time
[17] Wei Q, Wang Z, Gu Y, Dai T. Nuclear radiation detection based treatment applications. In: Proceedings of 2009 9th
on un-covered CMOS camera under static scene. Energy Sci International Conference on Information Technology and
Technol. 2017;51(1):175–9 (in Chinese). Applications in Biomedicine, 2009 November 4–7. Larnaca,
[18] Wei Q, Wang Z, Yao R, Gu Y, Dai T. Surveying ionizing Cyprus: IEEE; 2009. p. 1–5.
radiations in real time using a smartphone. Nucl Sci Tech. [28] Heal Force [cited 2020 Jan 10]. Available from: http://www.
2017;28(5):81–5. healforce.com/en/index.php?ac=article&at=read&did=482.
[19] Quick D. WikiSensor app turns an iPhone into a peripheral- [29] Cardoso JF. Blind signal separation: Statistical principles. Proc
free radiation detector [cited 2019 Oct 21]. Available from: IEEE. 1998;86:2009–25.

You might also like