You are on page 1of 5

The 2017 ISEE

SIMULATION OF REAL-TIME HEART RATE AND
ELECTROCARDIOGRAPHY DETECT USING KINECT IN VIET
NAM
Tran Thanh Dang Khoa(1), Luong Hoang Phuc(2)
(1)
Bach Khoa University , Viet Nam
(2)
Bach Khoa University, Viet Nam
Email: 1411856@hcmut.edu.vn

need to create a program that can carry out similar
ABSTRACT
procedures is increasing. Since then, the idea for this
This paper presents the real-time, non - contact heart project is formed.
rate measurement through video which captures patient’s
In this project, the blood volume in the vein of each
face by Kinect sensor. Because of the limitations and
heartbeat is noted for achieving the goal of measuring
difficulties in technology and knowledge, this paper
heart rate. Considering that the face is the easiest to reach
regretfully ignores electrocardiography and focuses on
and identify, the amount of blood vein on the face is
heart rate measurement. The method is executed by
distributed quite a lot and concentrated compared to other
computing the amount of light that hemoglobin absorbs at
regions of the body so the face was chosen as the input for
the forehead of the face. First, the green channel of the
the project. This project is also based on the theory that
forehead is filtered. Then, green channel’s mean value is
with different number of hemoglobin count on the surface
calculated for each frame, converted to frequency. Fast-
will absorb a different amount of light, resulting in
Fourier Transform and Hamming window technique are
differences in the value of green channel. Then, a series of
applied next to detect pulse. The outcome result is strongly
computational systems are used to reach the final output
influenced by the light conditions and distance between
value of the heartbeat. Experimentation shows that the
the sensor and patient.
result of the system which this project achieves is quite
KEYWORDS: pulse, face detection, Kinect sensor, accurate and valuable to continue to grow and expand.
opencv, openni, heart rate.
The paper has 5 pages in total.
1. INTRODUCTION
2. EXPERIMENT
Heart rate is the number of heart beats per minute. It
depends on the individual, age; body weight; Active status 2.1 Experiment apparatus
such as sitting or moving; diseases; even the air The project is built on Visual Studio 2013 Ultimate as
temperature can also affect the heart rate. In modern well as support libraries such as OpenCV, OpenNI,
medicine, heart rate is a particularly important biological NiTe2. Code is implemented on C ++ platform. The input
factor and is frequently mentioned first when there are is a real-time video that captures the face of an object
pathological problems. It helps diagnose and evaluate the captured by a Kinect sensor or conventional camera.
condition of the patient. 2.2 Methods and ideas
Ever since, heart rates have been measured by 2.2.1 Face and forehead detection
classical methods that are difficult and inadequate, such as
hand-held pacemakers or stethoscopes, based on the The first step of this topic is face detection. This is
experience of the doctor. For old methods requiring close considered to be the easiest and most important step. The
contact with patients, if there are cases of needle method used to detect faces here is the Haar-like feature
aspiration, heart rate measurement requires multiple assist based classifier. Haar-like is currently the most commonly
devices to prevent infection. On the other hand, with the used method for ease of use and operation. Basically, this
method divides the frame into multiple rectangles and then
increasing level of computer science and technology, the

1

it is split into 3 separate color channels: Red. Hamming window is given by: page 22. works best with it (has magnitude power of 2). The figure 4 shows the result of this effort After the forehead area is found and created a new ROI. “It is constant part and applied filtering. Filtered data set 2. the strongest periodic signal in frequency between detrending process is combined.if filtered output Fig 1. After the face is detected. better to have an even number of the sample because FFT Hamming window – a mathematical function that is zero. it will not be further considered in the following features. a region of interest (ROI) is created around the face. 2𝑖 𝑊[𝑖] = 0. camera vibration and other sources. Firuge. Signal has gone through noise handling Continuity. in this case it is 45-180 bps is considered as heart rate “ – Reference [1]. Fig 3. the Fast Fourier Transform (FFT) is applied to turn the filtered data to Frequency Domain. 2). The forehead area is defined through its location relative to the ROI. rate should be the only really periodical source in the Furthermore.46cos⁡( ) 𝑁−1 2 . the data obtained is de-trended. Interpolation. Therefore the green channel is chosen to get the signal for the next step by calculating its mean value. removed its scientific report that this project has referred to. All cascade features must be passed in order to perform a successful detection. Figure 5 is an example of this step.3 Acquiring heart rate To the final steps to achieve heart rate. For the filter step. one more signal. Hemoglobin exists in the blood that absorbs the green light so the change in the green channel is most apparent (as shown in figure. it was considered as 2. Green and Blue. Fig 4. to increase the accuracy. Next. According to the study of a Next.2. if it does not meet requirements. The cascade classifier includes many individual Haar features. We also need to filter out the noise coming from factors such as: movement of objects. the theory that different areas have different brightness levels. Each area is compared to current feature.2. An successful face and forehead detection value is outside defined interval. regions. Since heart valued outside of some chosen interval – is used. paragraph 2. also known as a The Figure 3 illustrates one sample of a set of data cascade classifier.2 Data processing product of one time artifact and output value is set to 0. a pre-built paradigm. The following approach is taken for handling and eliminating the external elements .1 demonstrates the outcome of this step.54 − 0. The 2017 ISEE differentiates the regions by the pixel mean of those For N is the total amount of points. Visible light absorption by hemoglobin frame rate of the camera. is applied to face recognition based on that has been filtered by metioned methods. The number of samples taken into the FFT is based on the Fig 2.

ANALYSIS In order to create a more user-friendly outcome. The program was tested on different subjects under different conditions as shown below: Figure 8. The data after FFT is applied Here is the equation for this step: 𝑓𝑝𝑠 𝐹𝑟𝑒𝑞𝑠 = 𝑖𝑛𝑑𝑒𝑥 ∗ 𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑙𝑎𝑡𝑒𝑑𝑆𝑖𝑧𝑒 ∗ 2 And finally. Subject 2 3 . Subject 1 ∑𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑟𝑙𝑎𝑡𝑒𝑑𝑆𝑖𝑧𝑒 𝐹𝑟𝑒𝑞𝑠 ∗ 60 𝐻𝑅 = ⁡ 𝐼𝑛𝑡𝑒𝑟𝑝𝑜𝑟𝑙𝑎𝑡𝑒𝑑𝑆𝑖𝑧𝑒 In conclusion. The 2017 ISEE Figure 5.the process’s chart 3. a user interface (GUI) has been created. Figure 6. the equation for the heart rate: Figure 7. the process is demonstrated as the chart in Figure 6.

Firstly. If the conditions are met. In the aspect of camera. we would like to thank Mr. and immense Heart rate knowledge. we must express our very Table 1. His guidance helped us in all the time of acquired research and writing of this thesis. time 17s 40s 74s  Create a multi-platform application for (second) smart phones. In addition. light condition appears to be the next important factor when it exerts a strong influence on the time it takes to produce accurate results. Test partitions profound gratitude to our family for supporting Based on the tests. the output value depends Gender Male Male Female significantly external condition. CONCLUSIONS In general. execution  Integrate with auto-save in online patient Measurement library. the Condition fluorine light bulbs daylight proposed directions are: light bulb  Full functioning Kinect mode in real-time. the result still has many limitation. the critical drawback is that Kinect heart rate measurement can only be performed by indirect method. for his insightful comments. Which means Figure 9.4% bound Last but by no means least. far distance but requiring short time. Secondly. the Kinect sensor has shown its qualities to be better than a conventional camera by showing the best result in bad conditions such as unstable light sources. with acceptable tolerances (< 7%). Dang Thanh Tin for the continuous support of our conditions glasses research.2% 5. the program functions properly throughout our research. for his patience.7m 0.7m  Add light adjustment program. Subject 3 that in order to use Kinect sensor. Distance to 1m 0. conventional camera and video inputs. improve face camera detection speed. Time of  Add in more non-contact sensor such as 17:20’ 20:25’ 20h25’ temperature and blink counter. the results are fairly accurate and valuable in medical. Input sources Kinect UVC UVC Primesense ACKNOWLEDGEMENT webcam webcam sensor We wish to express our sincere gratefulness to Additional With None none Dr. the heart rate measurement then can be functioned. After the Subject no. 1 2 3 conversion is done. a recording of the subject in . but also for the hard Heart rate 77 81 92 question which incented us to widen our research from Error various perspectives. The 2017 ISEE as lighting.ONI format must firstly be captured by this device. distance and camera quality. 1. Major errors come from important parameters that need to be considered such 4 . 4. sometime indicates instability. motivation. Light Single Fluorine LED 15W For more over research and development. However. then converted to . 76 86 87 from the program Besides.AVI format. the output result achieved a central part of the subject: measuring the heart rate with Kinect sensor. Huynh Xuan Canh.3% 6.

Available: https://dotblogs. 2013. “Detecting Pulse from Head Motions in Video”. Clifford. [Online]. [4] Daniel Lélis Baggio and Shervin Emami. “OpenNI Cookbook”. Malardelan University. [Accessed: 15 – Mar – 2017]. [Accessed: 20 – Apr – 2017]. “Windows Forms Application”.com/Articles/741055/Vide o-Capture-using-OpenCV-with-VCplusplus.youtube. [Online]. [8] Michael May. [Online]. [2] Guha Balakrishnan and Fredo Durand and John Guttag. [Accessed: 25 – Apr – 2017]. Available: https://www. [9] dotBlogs. [7] ProgrammingKnowledge. [Online]. [6] Soham Gandhi. “Webcam Pulse Detector”. [Accessed: 13 – Mar – 2017]. “Video Capture using OpenCV with C++”. MIT CSAIL. Cross College. 5 . “Intelligent system for monitoring physiological parameters using camera”. [5] Gari D.com. “Signal Processing Methods for Heart Rate Variability”. [3] Soroush Falahati. Available: https://www.com/watch?v=YR6fxe1wa8g& list=PLS1QulWo1RIZz6uDid--I09EOImRmPHS0. Packt Publishing. 2012.tw/v6610688/2014/03/02/open ni_oni_format_file_to_avi_opencv.com/maym86/webcam_pulse_detector . Available: https://github. 2012. The 2017 ISEE REFERENCES [1] Kh Nafis Karim. “Mastering OpenCV with Practical Computer Vision Projects”.codeproject. St. “OpenNI oni format file to avi”. Packt Publishing.