Heart Rate Measurement Using Facial Video

A
Dissertation Report
On
Heart Rate Measurement System using Facial Video

Processing
Submitted in Partial Fulfillment of the

Requirements For the Degree of
MASTER OF ENGINEERING
IN
Computer Engineering
By
Yuvraj D. Patil Exam Seat No: 9535
Under the guidance of
Mrs. G. J. Chhajed
Department of Computer Engineering
Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering

and Technology
(Formerly Vidya Pratishthan’s College Of Engineering)
Vidyanagari,Bhigwan Road, Baramati – 413133, Dist- Pune (M.S.)
Savitribai Phule Pune University, Pune (Maharashtra)
INDIA
June-2017
DEPARTMENT OF COMPUTER ENGINEERING
Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering
and Technology
(Formerly Vidya Pratishthan’s College Of Engineering)
Baramati – 413133, Dist- Pune (M.S.)
CERTIFICATE
This is to certify that the dissertation report entitled
Heart Rate Measurement System using Facial Video
Processing
Submitted by
Yuvraj D. Patil Exam Seat No. : 9535
is a bonafide work carried out by his under the supervision of Mrs. G. J.

Chhajed and it is submitted towards the partial fulfillment of the requirement of
Savitribai Phule Pune University for the award of the degree of Master of Engi-
neering (Computer Engineering).
Mrs. G. J. Chhajed Prof. S. A. Shinde Prof. Mrs. S. S. Nandgaonkar

Internal Guide PG Co-ordinator Head of Department
Dept. of Computer Engg. Dept. of Computer Engg. Dept. of Computer Engg.
Dr. D. B. Hanchate Dr. M. G. Devamane

Institute Incharge Principal
Internal Examiner External Examiner

Place: VPKBIET, Baramati.
Date:
Certificate by Guide
This is to certify that Yuvraj D. Patil has completed the dissertation work
under my guidance and supervision and that, I have verified the work for it’s
originality in documentation, problem statement, implementation and results pre-
sented in the dissertation. Any reproduction of other necessary work is with the
prior permission and has given due ownership and included in the references.
Place: VPKBIET, Baramati. Signature of Guide

Date: Mrs. G. J. Chhajed
ii
Acknowledgements
This is to acknowledge and thank all the individuals who played defining
role in shaping this dissertation report. Without their constant support, guidance
and assistance, this report would not have been completed alone.
I take this opportunity to express my sincere thanks to my guide Mrs. G.

J. Chhajed for her guidance, support, encouragement and advice. I will forever
remain grateful for the constant support and guidance extended by my guide in
making this dissertation work.
I wish to express my sincere thanks to Prof S. S. Nandgaonkar, (Head,

Department of Computer Engineering) at VPKBIET, Baramati, Prof S. A.
Shinde (PG coordinator) and all the departmental staff members for their sup-
port. I would also like to express my deep gratitude to our Hon’ble Principal who
provides good opportunities for all of us.
Yuvraj D. Patil
ME (Computer Engg.)
Exam Seat No. 9535
VPKBIET, Baramati
iii
Publications based on this Thesis
1. “Survey on Non-Invasive Methods of Heart Rate Measurement” published in

International Journal of Science and Research (IRJET) Volume 4, Issue 5, July
2017.
2. “Heart Rate Measurement System using Facial Video Processing” accepted in

International Journal on Emerging Trends in Technology (IJETT) Volume 4,Spe-
cial Issue, May 2017.
3. “Heart Rate Measurement System using Facial Video Processing” presented in

cPGCON-16, Computer Engineering- PG Conference, (Savitribai Phule Pune
University),Organized by Department of Computer Engineering, SITRC, Nashik.
iv
Abstract
In recent years, new technologies used for providing clinical health care remotely
appears to be new fields like a telemedicine huge advancements. New ways for
monitoring patients are automatically developed, as well as techniques for mea-
suring physiological parameters out of the hospital. One of these parameters
is the heart rate and usually used by medical professionals to assist in diagno-
sis. Measuring heart rate there are different standard techniques available such as
Electrocardiogram which is expensive and discomfort. Another commercial device
is oximetry sensor that needs attachment to fingertips, is also inconvenient.HR is
one of the simplest cardiovascular parameter and identifiable as an independent
risk parameter.
During the cardiac cycle, volumetric changes in the facial blood vessels
modifies the path length of the incident ambient light. The subsequent changes in
amount of reflected light indicate the timing of cardiovascular events. By record-
ing facial image region with a webcam, the RGB color sensors pick up a mixture
of the reflected Photoplethysmographic (PPG) signal along with other sources of
fluctuations in light due to artifacts such as motion and changes in ambient light-
ing conditions. These observed signals from the red, green and blue color sensors
are used for further process.
Green band signals are extracted by applying Signal Separation algorithm

techniques on these signals. In this contribution Independent Component Analy-
sis (ICA) algorithm is used for getting accurate results.
After extracting green band signals, select Region of Interest (ROI) by using
bucketing technique and finally obtain Inter Beat Interval (IBI) and calculate heart
rate.
1
Contents
Abstract 1
List of Figures 6
List of Tables 7
1 Introduction 9
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2 Existing Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.4 Flow Of Proposed System . . . . . . . . . . . . . . . . . . . . . . . 13
1.5 Features of Proposed System . . . . . . . . . . . . . . . . . . . . . . 13
1.6 Mathematical Model . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 Literature Survey 15
2.1 Advancements in noncontact, multiparameter physiological mea-
surements using a webcam . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Video-based heart rate measurement from human faces . . . . . . . 16
2.3 Dynamic heart rate measurements from video sequences . . . . . . 16
2.4 Comparison of Independent Component Analysis(ICA) algorithm
for Heart Rate Measurement Based on Facial Imaging . . . . . . . . 17
2.5 A computationally efficient heart rate measurement system using
video cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.6 MA Reduction by using Traditional Approaches . . . . . . . . . . . 18
2.7 MA reduction using Ring Sensor . . . . . . . . . . . . . . . . . . . . 19
2.8 Spatio-temporal filtering of thermal video sequences for heart rate
estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.9 Mobile Cardiac Pulse Measurements . . . . . . . . . . . . . . . . . 19
2.10 Heart Rate, Respiration Rate and Oxygen Saturation Level Moni-
toring from Video Recording by Smart phone camera . . . . . . . . 20
3 Software Requirement and Specification 21
3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Key Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3 Software to be used: . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4 Requirements Specification: . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 Functional Requirements: . . . . . . . . . . . . . . . . . . . 23
3.4.2 External Interface Requirements: . . . . . . . . . . . . . . . 23
3.4.3 Detailed Non-Functional Requirements: . . . . . . . . . . . . 23
4 System Design 25
4.1 Data Flow Diagram (DFD) . . . . . . . . . . . . . . . . . . . . . . 25
4.1.1 DFD Level-0 . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.2 DFD Level-1 . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.2 UML Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.1 Use Case Diagram . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.2 Class Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.2.3 Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . 28
4.2.4 State Transition Diagram . . . . . . . . . . . . . . . . . . . 28
4.2.5 Activity Diagram . . . . . . . . . . . . . . . . . . . . . . . . 29
5 Technical Specifications 30
5.1 Windows 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.2 Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5.3 Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6 Project Estimate, Schedule 32

6.1 Gantt Charts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.1.1 Planner Chart . . . . . . . . . . . . . . . . . . . . . . . . . . 32
6.1.2 Gantt Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
6.1.3 Test Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.2 Project Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.2.1 COCOMO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7 System Architecture 38
7.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
7.2 Photoplethysmography . . . . . . . . . . . . . . . . . . . . . . . . . 38
7.3 Extracting of BVP from Webcam Recordings . . . . . . . . . . . . . 40
7.4 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
7.5 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3
7.5.1 Architectural Block Diagram . . . . . . . . . . . . . . . . . . 43
7.5.2 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . 44
7.5.3 Data Preprocessing . . . . . . . . . . . . . . . . . . . . . . . 45
7.5.4 Parameter Extraction . . . . . . . . . . . . . . . . . . . . . . 49
8 Software Testing 51
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
8.2 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8.2.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
8.2.2 Fields in a Testcase . . . . . . . . . . . . . . . . . . . . . . . 52
8.2.3 Modules for Testing . . . . . . . . . . . . . . . . . . . . . . . 53
8.3 White Box Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
8.4 Black Box Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
9 Results and Discussion 57

9.1 Result snapshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
9.1.1 Graphical User Interface (JAVA And MATLAB): . . . . . . 57
9.1.2 Graphical User Interface (Python): . . . . . . . . . . . . . . 59
9.2 Result Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
9.3 Result Comparison Bar and Line Graph . . . . . . . . . . . . . . . 61
10 Conclusion and Future Work 62

10.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
10.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
A Publications 66
4
List of Figures
1.1 Examples of ECG devices . . . . . . . . . . . . . . . . . . . . . . . 11

1.2 Examples of pulse oxiymetry devices . . . . . . . . . . . . . . . . . 12
4.1 DFD Level-0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.2 DFD Level-1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 Use Case Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 Class Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.5 Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.6 State Transition Diagram . . . . . . . . . . . . . . . . . . . . . . . 28
4.7 Activity Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.1 Planner chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

6.2 Gantt Chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
7.1 Photoplethysmography imaging techniques . . . . . . . . . . . . . . 39

7.2 Recovery of the BVP waveform. (a) Face within the first video
frame is automatically detected to locate the ROI. (b) ROI is de-
composed into red, green, and blue channels for each frame and
spatially averaged to form (c) the raw signals. After the raw sig-
nals are detrended and normalized, ICA is applied to separate three
independent sources. In this example, the BVP is visible in the sec-
ond source signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
7.3 Modules in Proposed System . . . . . . . . . . . . . . . . . . . . . 43
7.4 Architectural Block Diagram . . . . . . . . . . . . . . . . . . . . . . 43
7.5 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
7.6 RGB traces Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . 46
7.7 Bucketing Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
9.1 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . 57

9.2 Frame Extracted Folder Window . . . . . . . . . . . . . . . . . . . 58
9.3 Graphical User Interface (JAVA and MATLAB) . . . . . . . . . . . 58
9.4 Graphical User Interface(PYTHON) . . . . . . . . . . . . . . . . . 59
9.5 Graphical User Interface and Output Graph . . . . . . . . . . . . . 59
9.6 Result Comparison Bar Graph . . . . . . . . . . . . . . . . . . . . . 61
9.7 Result Comparison Line Graph . . . . . . . . . . . . . . . . . . . . 61
A.1 Survey Paper Certificate . . . . . . . . . . . . . . . . . . . . . . . . 67

A.2 cPGCON Certificate . . . . . . . . . . . . . . . . . . . . . . . . . . 68
A.3 cPGCON Evaluation Sheet . . . . . . . . . . . . . . . . . . . . . . . 69
A.4 Hall Ticket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
6
List of Tables
6.1 Project Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

6.2 Constant Values for COCOMO Model . . . . . . . . . . . . . . . . 36
6.3 Size (Number of Lines) of Project Module Wise . . . . . . . . . . . 36
8.1 Capture Video and Frames Extraction . . . . . . . . . . . . . . . . 53

8.2 Obtain ROI And RGB Extraction from Frames . . . . . . . . . . . 53
8.3 Signal Separating using ICA and Signal Filtering using FFT . . . . 53
8.4 Calculate Inter Beat Interval and Calculate Hear Rate . . . . . . . 53
9.1 Comparison of actual result and Experimental Result . . . . . . . . 60
A.1 Publication on This Thesis . . . . . . . . . . . . . . . . . . . . . . . 66
7
Abbreviations
PPG Photoplenthesmography imaging technique

HR Heart Rate
RR Respiration Rate
MA Motion Artifacts
ICA Independent Component Analysis
ROI Region Of Interest
RGB Red Green Blue
IBI Inter Beat Interval
BVP Blood Volume Pulse
FFT Fast Fourier Transform
RMSE Root Mean Square Error
8
Chapter 1
Introduction
1.1 Background
Now a day’s the risk factors such as obesity, hypertension and heart diseases are
increasing at a tremendous rate. People are suffering from such health disorders
due to unhealthy eating habits, sedentary lifestyle, lack of daily exercise and lack of
proper knowledge and awareness about health related factors. The percentage of
hypertension and diabetes in early age (i.e. between 25-35 years) is also increasing.
Heart disease and stroke can affect anyone regardless of age, race, ethnicity, sex
and income level. This is harmful or can cause severe heart disease and can
increase risk of cardio respiratory failure if doesn’t handled properly.
Regular and non-invasive assessments of cardiovascular function are important
in surveillance for cardiovascular catastrophes and treatment therapies of chronic
diseases. Resting heart rate, one of the simplest cardiovascular parameters, has
been identified as an independent risk factor (comparable with smoking, dyslipi-
demia or hypertension) for cardiovascular disease. Currently, the gold standard
techniques for measurement of the cardiac pulse such as the electrocardiogram
(ECG) require patients to wear adhesive gel patches or chest straps that can cause
skin irritation and discomfort. Commercial pulse oximetry sensors that attach to
the fingertips or earlobes are also inconvenient for patients and the spring-loaded
clips can cause pain if worn over a long period of time. The ability to monitor
a patient’s physiological signals by a remote, non-contact means is a tantalizing
prospect that would enhance the delivery of primary healthcare. E.g the idea of
performing physiological measurements on the face was first postulated by Pavlidis
and associates and later demonstrated through analysis of facial thermal videos.
Although non-contact methods may not be able to provide details concerning car-
diac electrical conduction that ECG offers, these methods can now enable long-
9
Chapter 1. Introduction
term monitoring of other physiological signals such as heart rate or respiratory

rate by acquiring them continuously in an unobtrusive and comfortable manner.
The use of PPG, a low cost and non-invasive means of sensing the cardiovas-
cular pulse wave (also called the blood volume pulse) through variations in trans-
mitted or reflected light, for non-contact physiological measurements has been
investigated recently. This electro-optic technique provides valuable information
about the cardiovascular system such as heart rate, arterial blood oxygen satu-
ration, blood pressure, cardiac output and autonomic function. Typically, PPG
has always been implemented using dedicated light sources (e.g. red and/or infra-
red wavelengths), but recent work has shown that pulse measurements can be
acquired using digital camcorders/cameras with normal ambient light as the illu-
mination source. However, all these previous efforts lacked rigorous physiological
and mathematical models amenable to computation; they relied instead on manual
segmentation and heuristic interpretation of raw images with minimal validation
of performance characteristics. Furthermore, PPG is known to be susceptive to
motion-induced signal corruption and overcoming motion artifacts presents one of
the most challenging problems. In most cases, the noise falls within the same fre-
quency band as the physiological signal of interest, thus rendering linear filtering
with fixed cut-off frequencies ineffective. In order to develop a clinically useful
technology, there is a need for ancillary functionality such as motion artifact re-
duction through efficient and robust image analysis.
The HR of a person indicates the rhythm at which the heart is working and
it depends on several factors, such as age, gender, illnesses or physical activity.
It can be measured at any place of the body at which an artery is close to the
surface, which is where the pulse is detected.
Heart Rate Measurement System 10 VPKBIET-Baramati

using Facial Video Processing
1.2 Existing Techniques

Currently, the gold standard techniques for measurement of the cardiac pulse are.
1. Electrocardiogram (ECG):
The electrocardiogram (ECG or EKG) is a noninvasive routine examina-
tion of the electrical activity of the heart that is used to reflect underlying
heart conditions. Regular heart movements are controlled by a complex set
of electrical impulses that direct the upper and lower heart chambers to con-
tract and relax rhythmically. When these electrical activities are interrupted
or misguided, this arrhythmia can be symptoms of heart diseases. Further-
more, the impulses can also show signs of structural or metabolic changes
like enlargement or hypoxia of cardiac muscle.
Figure 1.1: Examples of ECG devices
2. Oximerty Sensors:
Pulse oximetry is the non-invasive measurement of the oxygen saturation
(SpO2). Oxygen saturation is defined as the measurement of the amount of
oxygen dissolved in blood, based on the detection of Hemoglobin and Deoxy-
hemoglobin. Two different light wavelengths are used to measure the actual
difference in the absorption spectra of HbO2 and Hb. The bloodstream is
affected by the concentration of HbO2 and Hb, and their absorption coeffi-
cients are measured using two wavelengths, one is 660 nm (red light spectra)
and another one 940 nm (infrared light spectra). Deoxygenated and oxy-
genated hemoglobin absorb different wavelengths.

Figure 1.2: Examples of pulse oxiymetry devices
1.3 Motivation
Now a day’s number of heart patients is increasing at tremendous rate. Symp-
toms like Obesity, lack of awareness about health, hypertension are observed even
in small ages.
Existing techniques such as ECG and pulse oxymeter have limitations such
as invasiveness, higher cost, bulkiness, time consuming, discomfort to the patient
etc.
Percentage of Computer techniques are used is also increased. It does not
require extra hardware and can act as take anywhere physiological monitor. There
is need of noninvasive, accurate, less costly physiological parameter monitoring
system which can measure vital parameters.

1.4 Flow Of Proposed System

There are three main steps.
Step 1: Data Collection module records the PPG videos by using the webcam
and light.
Step 2: Data preparation does the process of videos for motion artifact re-
duction and these noise free videos are used for parameter extraction.
Step 3: Parameter Extraction module extracts the values of heart rate.
1.5 Features of Proposed System

1. Videos are used for extracting PPG signals. Face is used skin part. Webcam
is used as a component of video recording.
2. Independent Component Analysis algorithm is used as a for motion artifacts

reduction.
3. Beat-beat interval computing algorithm, standard deviation, mean intensi-

ties are used for parameter extraction.
1.6 Mathematical Model

Let, V is the set of video frames in .avi format.
V = {F 1, F 2, F 3, ..., F n}
where, F1,F2,F3,...,Fn are the frame and n is the number of frames in video V.
let, F = {{R1, R2, ..., Rn} , {G1, G2, ..., Gn} , {B1, B2, ..., Bn}}
where,
R1,R2,...,Rn are the frame containing only red band.
G1,G2,...,Gn are the frame containing only green band.
B1,B2,...,Bn are the frame containing only blue band.
let, F=A X S where, A is the Mixing Matrix and S is the Motion Analysis reduced
frame signal which is to be obtained.
N
X
F = ai × s i (1.1)
i=1

let,
S=W X F
Where
W is the inverse matrix of A
Separate S into 3 bands i.e. RGB

Chapter 2
Literature Survey
2.1 Advancements in noncontact, multiparame-

ter physiological measurements using a we-
bcam
Poh et al. developed a method for measuring multiple physiological parameters
using a basic webcam. The blood volume pulse (BVP) signals were recovered from
the facial region of the subjects. Independent component analysis (ICA) was used
to separate the sources from the colour channels in the video recordings. In con-
trast to correlation-based transformation such as Principal Component Analysis
(PCA), ICA does not only de-correlate the signals, but also reduces higher-order
statistical dependencies. The red, green, and blue (RGB) components in the video
recordings are actually the sensors or mixture of the reflected plethysmographic
signals and other sources (as well as the artefacts).
• Advantage:
1. Noninvasive method for detecting heart rate. It doesn’t require any

contact with the skin for monitoring heart rate unlike Existing methods
i.e.Commercial pulse oximetry sensors.
2. It doesn’t require any extra hardware components.Such a technology
would also minimize the amount of cabling and clutter associated with
neonatal ICU.
3. Natural source light i.e. sun light is used for illuminating the skin.
• Limitations:
Variations in sunlight can cause decreasing SNR.
15
Chapter 2. Literature Survey
2.2 Video-based heart rate measurement from

human faces
In addition to the method using ICA, Pursche et al. concluded that centre
of the face region provides better information compared to the other parts of the
face region. They concluded that the power-spectrum-analysis algorithm can also
give similar results compared to the peak-detection algorithm.
• Advantage:
1. It is a first attempt for calculating heart rat, respiration rate and heart
rate variability in a single non contact,noninvasive system.
2. It doesn’t require any extra hardware components.
• Limitations:
This technology uses the webcam available as inbuilt feature with laptop.But
the quality of videos can undergo changes due to different resolution of a
camera.
2.3 Dynamic heart rate measurements from video

sequences
Yong-Poh Yu the proposed model to estimate the dynamic heart rate mea-
surements is discussed together with how the independence of the ICA sources
is established.sign of the independence of the ICA sources determined from the
mutual information establishes the minimum video duration which gives the most
accurate heart rate estimation is elaborated. The significance of the video duration
and its relationship to the accuracy of the heart rate estimation is also discussed.
• Advantage:
1. Minimum video processing required for heart rate measurement.

• Limitations:
Variations in sunlight can cause decreasing SNR.

2.4 Comparison of Independent Component Anal-

ysis(ICA) algorithm for Heart Rate Mea-
surement Based on Facial Imaging
L.Septiana , F.Hariyanto and K.P.Lin compare two methods which are deals
with the heart rate measurement by performing automatic face tracking and blind
source separation of three color channel into independent components. A class of
the so-called Independent Component Analysis (ICA) represents a powerful tool
for such a detection. Various ICA algorithms have been introduces in the liter-
ature; therefore there is a need to compare these methods. In this contribution,
two of the most common ICA methods are studied and compared to each other as
regarding their ability to recover the independent source signal from normalized
RGB of the facial image. These are the Joint Approximate Diagonalization of
Eigen matrices (JADE), and the Second Order Blind Identification (SOBI). These
two algorithms have been applied to the same data set of RGB traces then compare
with commercially calibrated BPV sensor. Both two methods have given approx-
imately consistent results. However SOBI method has shown better accuration of
heart rate measurement over JADE.

2.5 A computationally efficient heart rate mea-

surement system using video cameras
Syed Muhammad Imaduddin, Yaseen Athar proposed a new algorithm for heart
rate measurement.The algorithm which are presently used These algorithms use of-
fline, computationally involved techniques such as Independent Component Anal-
ysis (ICA) or Principal Component Analysis (PCA) which render the algorithms
unfeasible for implementation on real-time embedded systems. Experiments con-
ducted by author to find an optimal colorspace for measuring the heart rate.Author
subsequently used a novel means of optical filtering of this colorspace to develop
an accurate realtime algorithm, without the need for ICA or PCA, using ordinary
standard definition (SD) web cameras.
• Advantage:
1. Minimum video processing required for heart rate measurement.

4. Time Required for Signal separating get reduced.
• Limitations:
Motion artifact is not handled properly.
2.6 MA Reduction by using Traditional Approaches

Rajet Krishnan, (Student Member, IEEE), Balasubramaniam Bala Natarajan,
(Senior Member, IEEE), and Steve Warren, (Member, IEEE) proposed Two-Stage
Approach for detection and reduction of motion artifacts in photoplethysmo-
graphic data in 2010. They present novel and consistent techniques to detect
the presence of motion artifact in PPGs given higher order statistical information
present in the data. J. G. Webster, M. R. Neuman and N.Wang, K. W. Chan
and Y. T. Zhang, J. Lee, W. Jung, I. Kang, Y. Kim, and G. Lee proposed that
The moving average (MA), adaptive, and multirate filtering techniques for MA
reduction.
• Limitations:
These techniques have limited scope and not suitable for estimation of every
physiological parameters.

2.7 MA reduction using Ring Sensor

Sokwoo Rhee, Boo-Ho Yang, and Haruhiko Harry Asada, (Associate Member,
IEEE proposed) the sensor, called a "ring sensor", which can be attached to a
finger base for monitoring beat-to-beat pulsation, and the data is sent to a host
computer via a radiofrequency transmitter. They published this technique in their
IEEE paper named as ”Artifact-Resistant Power-efficient Design of Finger-Ring
Plethysmographic Sensors[july 2001]”.
• Advantage:
1. Motion artifact reduction is handled.

2. The consumption of battery power is minimized
• Limitations:
Extra hardware is required to capture PPG signals.
2.8 Spatio-temporal filtering of thermal video se-

quences for heart rate estimation
Kian Hamedani developed a system in which thermal imaging video is cap-
tured. Consider the ROI on that captured video and processed that ROI and
calculate the heart rate.In their method video is captured by thermal camera.
These raw thermal images are considered as an input data. In second step mo-
tions are tracked and blood vessels are registered. After blood vessels registration
FFT analysis is done. Adaptive function is estimated on that analysis and calcu-
lates the cardiac pulses
2.9 Mobile Cardiac Pulse Measurements

In 2014 Haiying Xia developed a system in which smart phone camera is used
as a photo detector and natural light is used as a source of light. In additional
author capture a video using smart phone camera and send it to server through
internet and at server side video is processed and calculate heart rate. The result
is send back on smart phone.

• Advantage:
1. Non-Invasive and contact less heart rate measurement system.

2. No extra hardware required.
• Limitations:
Low camera sampling rate of mobile phone and Motion artifacts can cause
error.
2.10 Heart Rate, Respiration Rate and Oxygen

Saturation Level Monitoring from Video Record-
ing by Smart phone camera
In 2014 a team of Shruti Madan Kshirsagar, Gyankamal J. Chhajed developed
a smart phone application that can measure heart rate, heart rate, respiration rate
and blood oxygen saturation using the phone’s built in camera as a photo detector
and LED flash light as a light source. Author capture video of thumb with flash
light, apply ROI on that video frames. Captured video is processed by using ICA
and FFT filtering technique and measure heart rate.
• Advantage:
1. The system can monitor HR, RR and SpO2 level on a single device and
it is Non-Invasive and less costly.
2. No extra hardware required.
3. It acts as a “take anywhere “physiological monitor.
4. It can be utilized for personal and clinical use.
• Limitations:
1. Low camera sampling rate of mobile phone and

2. Motion artifacts can cause error.

Chapter 3
Software Requirement and

Specification
3.1 Purpose
The tremendous increase in number of heart patients motivated scientists to de-
velop new techniques for physiological parameter monitoring. Some applications
which are present in the market have some drawbacks as mentioned below.
1. Costly
2. Time Consuming
3. Can’t carry anywhere
4. ECG- adhesive gel patches or chest straps can cause skin irritation and
discomfort.
5. Oximeter sensors attached to the fingertips or earlobes are inconvenient for

patients and the spring-loaded clips can cause pain.
So in order to avoid these drawbacks, in this report new kind of model is presented
which will detect HR from video processing by using inbuilt webcam of a laptop
and smart phone camera.
3.2 Key Benefits

1. User Friendly.
2. Affordable.
21
Chapter 3. Software Requirement and Specification
3. No extra hardware settings should be required.
4. Reduction of motion artifacts which will cause errors in final output.
5. Allow good GUI.
6. Generate accurate results.
3.3 Software to be used:

1. JAVA (JDK 1.6.0 or above):
Java is a computer programming language that is concurrent, class-based,
objectoriented, and specifically designed to have as few implementation de-
pendencies as possible. It is intended to let application developers ”write
once, run anywhere” (WORA), meaning that code that runs on one platform
does not need to be recompiled to run on another.
JAVA is used for main implementation purpose and video processing.
2. MATLAB 7.0:
MATLAB, which stands for MATrix LABoratory, is a software package de-
veloped by MathWorks, Inc. to facilitate numerical computations as well as
some symbolic manipulation.
MATLAB will be used for some complex signal processing algorithm.
3. Python 36:
Python is a multi-paradigm programming language: object-oriented pro-
gramming and structured programming are fully supported, and many lan-
guage features support functional programming and aspect-oriented pro-
gramming (including by metaprogramming and metaobjects (magic meth-
ods)). Many other paradigms are supported via extensions, including design
by contract and logic programming.
Python uses dynamic typing and a mix of reference counting and a cycle-
detecting garbage collector for memory management. An important feature
of Python is dynamic name resolution (late binding), which binds method
and variable names during program execution.
The design of Python offers some support for functional programming in
the Lisp tradition. The language has map(), reduce() and filter() functions;
list comprehensions, dictionaries, and sets; and generator expressions. The

standard library has two modules (itertools and functools) that implement
functional tools borrowed from Haskell and Standard ML.
3.4 Requirements Specification:

3.4.1 Functional Requirements:
The proposed project, physiological parameter monitoring using videos provides
following functionality to user.
Functional requirements:
1. Record videos
2. System will process the videos for parameter extraction.
3. User will get the values of HR as an output from system.
3.4.2 External Interface Requirements:

System needs to have,
1. Java (JDK 1.6.0 or above)
2. Matlab 7.0
3. Python 36
4. Open CV 3.1.0
5. Ambient sunlight is needed to record good video signals.
3.4.3 Detailed Non-Functional Requirements:

These are requirements that are not functional in nature, that is, these are con-
straints within which the system must work.
• Performance requirements:
1. The system should give accurate values of all the three parameters.
2. If anyone of the parameter level drops below threshold value then the
proper advice should be given to user by generating alarm signal.
3. Most of the user requirements should be satisfied.

• Design Constraints:
Software constraint:
1. Platform : Windows 8 or higher

2. Coding Language : JDK 1.7 or more, Matlab, Python
3. IDE Tool:Eclipse
Hardware Constraints:
Laptop with inbuilt webcam and enough sunlight is needed for accurate
results.
• Software System Attributes:
1. Reliability: There were no explicitly stated reliability requirements.

2. Availability: The software uses system database, so that storage and
retrieval is easy.
3. Maintainability: The logic must accommodate addition of tools and
other requirements of user that has to be developed in the future.
4. Portability: This software uses java and MATLAB 7.0 components,
thus, it is portable.
5. User Friendliness: User must easily understand and use the utilities
provided on Speech emotion analysis.
6. Continuous Product Advancements: System developments are
based on user’s requirement and feedback.

Chapter 4
System Design
4.1 Data Flow Diagram (DFD)

4.1.1 DFD Level-0
Figure 4.1: DFD Level-0
25
Chapter 4. System Design
4.1.2 DFD Level-1
Figure 4.2: DFD Level-1

4.2 UML Diagrams

4.2.1 Use Case Diagram
Figure 4.3: Use Case Diagram
4.2.2 Class Diagram
Figure 4.4: Class Diagram

4.2.3 Sequence Diagram
Figure 4.5: Sequence Diagram
4.2.4 State Transition Diagram
Figure 4.6: State Transition Diagram

4.2.5 Activity Diagram
Figure 4.7: Activity Diagram

Chapter 5
Technical Specifications
5.1 Windows 7
Windows 7 is operating system for a personal and professional computer sys-
tem developed by Microsoft. Windows 7 was released to manufacturing on July
22, 2009. Windows 7 was primarily intended to be an incremental upgrade to the
operating system, intending to address criticisms faced by its predecessor. In just
six months, over 100 million copies had been sold worldwide, increasing to over
630 million licenses by July 2012, and a market share of 47.49 percentage as of
February 2014 according to Net Applications, making it the most widely used ver-
sion of Windows.Windows 7’s new features are advances in touch and handwriting
recognition, support for virtual hard disks, improved performance on multi-core
processors, improved boot performance, Direct Access, and kernelimprovements.
Windows 7 adds support for systems using multiple heterogeneous graphics cards
from different vendors.
5.2 Java
Java is a programming language originally developed by James Gosling at
Sun Microsystems (which has since merged into Oracle Corporation) and released
in 1995 as a core component of Sun Microsystem’s Java platform. The language
derives much of its syntax from C and C++, but it has fewer low-level facilities
than either of them. Java applications are typically compiled to byte code (class
file) that can run on any Java virtual machine (JVM) regardless of computer ar-
chitecture. Java is a general- purpose, concurrent, class-based, object-oriented
language that is specifically designed to have as few implementation dependencies
as possible. It is intended to let application developers “write once, run any-
30
Chapter 5. Technical Specifications
where” (WORA), meaning that code that runs on one platform does not need to
be recompiled to run on another. Java is, as of 2012, one of the most popular pro-
gramming languages in use, particularly for client-server web applications, with a
reported 10 million users. The Java Programming Language is a general-purpose,
concurrent, strongly typed, class-based object-oriented language. It is normally
compiled to the byte code instruction set and binary format defined in the Java
Virtual Machine Specification. Java language was designed to be small, simple,
and portable across platforms, operating systems, both at the source and at the
binary level, which means that Java programs (applet and application) can run
on any machine that has the Java Virtual Machine (JVM) installed.
5.3 Python
Python is a multi-paradigm programming language: object-oriented pro-
gramming and structured programming are fully supported, and many language
features support functional programming and aspect-oriented programming (in-
cluding by metaprogramming and metaobjects (magic methods)). Many other
paradigms are supported via extensions, including design by contract and logic
programming.
Python uses dynamic typing and a mix of reference counting and a cycle-
detecting garbage collector for memory management. An important feature of
Python is dynamic name resolution (late binding), which binds method and vari-
able names during program execution.
The design of Python offers some support for functional programming in the
Lisp tradition. The language has map(), reduce() and filter() functions; list com-
prehensions, dictionaries, and sets; and generator expressions. The standard li-
brary has two modules (itertools and functools) that implement functional tools
borrowed from Haskell and Standard ML.

Chapter 6
Project Estimate, Schedule
6.1 Gantt Charts

6.1.1 Planner Chart
Figure 6.1: Planner chart
32
Chapter 6. Project Estimate, Schedule
6.1.2 Gantt Chart
Figure 6.2: Gantt Chart

6.1.3 Test Plan

6.2 Project Estimation
Sr. Milestone Milestone Descrip-

Timeline Remark
No Name tion
01/08/2016 Detailed document
Requirement Complete Specifica-
01 To should be there for
Analysis tion Of System
10/09/2016 each requirement
Identify Mod- 10/09/2016
High Level De- Should decide interac-
02 ule,different Entities To
sign tion of module
& their relationship 25/09/2016
25/09/2016 Identify the user
GUI Design,program
03 Detailed Design To friendly nature of
specification etc
25/10/2016 GUI.
Code for the system 25/10/2016
Writing code for dif-
04 Build implementation of To
ferent module.
each module 15/01/2017
15/01/2017 Identify output of
Testing of each indi-
05 Unit testing To each module as per
vidual module
31/01/2017 require ment.
01/02/2017
Future Work Im- Design and implemen- Design system for fu-
06 To
plementation tation of future work ture changes .
28/02/2017
01/03/2017 Display the result in
Identify performance
07 Result Analysis To the form of graphical
of the system
15/03/2017 respresentation.
15/03/2017
Testing & De- Apply different testing
08 System level testing To
ployment strategies.
31/03/2017
Table 6.1: Project Estimation
6.2.1 COCOMO
The Constructive Cost Model (COCOMO) is an algorithmic software cost
estimation model developed by Barry W. Boehm. The model uses a basic regres-
sion formula with parameters that are derived from historical project data and
current project characteristics.
COCOMO consists of a hierarchy of three increasingly detailed and accurate

forms. The first level, Basic COCOMO is good for quick, early, rough order of
magnitude estimates of software costs, but its accuracy is limited due to its lack
of factors to account for difference in project attributes (Cost Drivers). Interme-
diate COCOMO takes these Cost Drivers into account and Detailed COCOMO

additionally accounts for the influence of individual project phases.
Basic COCOMO Model Formulae :
• Effort Applied (E) = ab (KLOC)bb [ man-months ]
• Development Time (D) = cb (Ef f ortApplied)db [months]
• People required (P) = Effort Applied / Development Time [count]
Where KLOC is the number of delivered lines (expressed in thousands) of code

for project. The coefficients ab , bb , cb and db are given in the following table:
Software Project ab bb cb db
Organic 2.4 1.05 2.5 0.38
Semi-detached 3.0 1.12 2.5 0.35
Embedded 3.6 1.20 2.5 0.32
Table 6.2: Constant Values for COCOMO Model
COCOMO Model for the System
Following table shows size of each module.
Sr. No Module Line of Code

1 Capture Video 470
2 Pre-Processing 1505
3 Results and graphs 550
Total 2525
Table 6.3: Size (Number of Lines) of Project Module Wise
1. Effort: Effort = ab *(KLOC)bb

=3.0*(2.525)1.12
=8.46 man / months
2. Schedule:= Effort/Size of Team

=8.46/2
=4.23 months (Approx)

3. Productivity:= KLOC/Effort
=2.525/4
=0.63125 KLOC/man per months

Chapter 7
System Architecture
7.1 Problem Statement

The main objective of proposed system is to develop an application which
can extract major physiological parameter HR. Proposed system records the PPG
videos by using one photo detector and one source of light for illuminating the
skin part. But even a slight movement of patient can cause great error in final
output. Hence the noise due to motion artifact should be removed to get accurate
results.
7.2 Photoplethysmography
Photoplethysmography (PPG) is a non-invasive method for studies of the blood
volume pulsations by detection and temporal analysis of the tissue back-scattered
or transmitted optical radiation. Blood pumping and transport dynamics can be
monitored at different body locations - fingertip, earlobe, forehead, forearm, etc.
with relatively simple and convenient PPG contact probes. The AC-component of
back-scattered PPG signals reliably reflects skin blood volume pulsations, there-
fore PPG technique has good potential to become a routine tool for express diag-
nostics and early screening of cardio-vascular pathologies, for self-monitoring at
home and in public facilities, as well as and for the tele-diagnostics via Internet
or LAN. The key factors that can affect time varying component of light intensity
are blood cells. Generally time varying component provides a signal proportional
to changes in skin blood volume but does not produce a quantitative measure.
The experimental setup for PPG requires two components.
1. Photo Detector : Used for capturing the PPG signals.
38
Chapter 7. System Architecture
2. Source of Light : Used for illuminating skin part.
Figure 7.1: Photoplethysmography imaging techniques
PPG for Heart Rate

The heart is one of the most important organs of the entire human system
since it is responsible for rhythmically pump blood all over the body enabling
the transport of vital nutrients and oxygen to the cells. A typical PPG signal is
made of large DC component that is due to a large part of light from the source
passing through skin-muscle-bone without coming into contact with blood vessels
at all and reaching the photo-detector, a very small component having very low
frequency due to light from the source passing though, apart from skin-muscle-
bone, the venous blood and a much smaller component at the frequency of the
heart beat due to light from the source also passing through arterial blood vessels.
Just after the systole, blood volume increases in the arteries thereby reducing the
received light intensity. During diastole, blood volume in the arteries decreases
and hence as increase in the light transmission. Thus the part of detected signal
due to the arterial blood appears pulsatile in nature at heart rate.
The pulsalite portion of PPG arises due to the light passing through arterial
blood and hence has information contained in the arterial blood flow; heart rate.
heart rate variability, respiration and blood pressure to name a few.

7.3 Extracting of BVP from Webcam Record-

ings
All the video and physiological recordings were analyzed offline using cus-
tom software written in MATLAB. Figure 7.2 provides an overview of the stages
involved in this approach to recover the BVP from the webcam videos. Open
Computer Vision library is utilized for automatically identify the coordinates of
the face location in the first frame of the video recording using a boosted cascade
classifier. The algorithm returned the x and y-coordinates along with the height
and width that define a box around the face and select the center 60% width
and full height of the box as the Region of Interest (ROI) for next subsequent
calculations.
The ROI is then separated into the three RGB channels [ Figure 7.2(b)] and
spatially averaged over all pixels in the ROIt o yield a red, blue, and green mea-
surement point for each frame and form the raw signals [ Fig. 7.2(c)] y1 (t), y2
(t), and y3 (t), respectively. The normalized raw traces are then decomposed
into three independent source signals using ICA [ Fig. 7.2(d)] based on the Joint
Approximate Diagonalzation of Eigenmatrices (JADE) algorithm. ICA is able to
perform motion artifact removal by separating the fluctuations caused predomi-
nantly by the BVP from the observed raw signals. However, the order in which
ICA returns the independent components is random. Thus, the component whose
power spectrum contained the highest peak was then selected for further analysis.

Figure 7.2: Recovery of the BVP waveform. (a) Face within the first video frame
is automatically detected to locate the ROI. (b) ROI is decomposed into red,
green, and blue channels for each frame and spatially averaged to form (c) the
raw signals. After the raw signals are detrended and normalized, ICA is applied
to separate three independent sources. In this example, the BVP is visible in the
second source signal.

7.4 Algorithm
1. Capture Video
2. Extract Photoplethysmographic (PPG) Signals From Video
3. Separate RGB raw traces from PPG signals.
4. Apply ICA signal separating algorithm on that raw traces.
5. Consider only green signal and discard red and blue signal.
6. Apply FFT filtering technique on that green signal.
7. Select Region of Interest (ROI) on green signal.
8. Obtain Inter Beat Interval (IBI).
9. Measure Heart rate using formula.
Heartrate = 60/M ean(IBI)
7.5 System architecture

In proposed system, there are three main modules.
1. Data Collection
2. Data Preprocessing
3. Parameter Extraction

Figure 7.3: Modules in Proposed System
7.5.1 Architectural Block Diagram
Figure 7.4: Architectural Block Diagram
Above block diagram shows that how actually heart rate measurement system is
worked. Detail step wise execution procedure is illustrated in the next section.

7.5.2 Data Collection

1. Video Capturing
For video capturing Open Computer vision library (Open CV) is utilized
to automatically identify the coordinates of the face location in the first
frame of video recording. The OpenCV Library is a way of establishing
an open source vision community that will make better use of up-to-date
opportunities to apply computer vision in the growing PC environment. The
software provides a set of image processing functions, as well as image and
pattern analysis functions. All videos were recorded in color (24-bit RGB
with three channels * 8 bits per channel) at 15 frames per second (fps) with
pixel resolution of 640 * 480 and saved in AVI format on the laptop.
2. Extracting PPG Signals From Video

A photoplethysmogram (PPG) is an optically obtained plethysmogram, a
volumetric measurement of an organ. Which capture cardiovascular system
movements. After extracting PPG signals from video mixture of RGB colors
is extracted.
3. Separate RGB raw traces from PPG signals

As a first approach, the region of the image where the face is located
is found using a face detection algorithm. Although use of fast and robust
approach, it has two main drawbacks: it limits the user motion to avoid false
negatives and includes non-skin pixels in the selected region. The first of
the drawbacks, the user motion problem, can be solved by adding a tracking
system which follows the position of the user face along consecutive frames,
even if it is partially occluded and cannot be detected. This allows more
position and pose variations to the user, increasing robustness and usability.
The inclusion of non-skin pixels in the selected region, such as hair or
background, is a strong drawback as well. By adding a skin pixel detector the
system can distinguish skin from non-skin pixels. This reduces the amount
of noise coming from non facial skin regions in the image and improves the
precision of the system. The final Region of Interest(ROI) selection system
is composed by a face detector.

Figure 7.5: Data Collection
The ROI was then separated into the three RGB channels and spatially aver-
aged over all pixels in the ROI to yield a red, blue and green measurement point
for each frame and form the raw traces.
x1 (t), x2 (t), x3 (t)
Then the raw RGB traces are normalized as follows:
xi (t) − µi
xi (t) =
σi
where
µi : M ean
σi : StandardDeviation
7.5.3 Data Preprocessing

Apply signal separating algorithm on that raw traces
As it is described in previous step, for each frame of the video sequence
obtain a ROI of facial skin pixels. These selected pixels contain information of
light fuctuations produced in the skin due to the variation in the amount of blood
present in the human face. To extract this information, the ROI is separated into
the three RGB (red, green, blue) channels and is spatially averaged over all skin
pixels.

Figure 7.6: RGB traces Recovery
Three color signals over time are obtained, as it is shown in Figure 7.6.
After that, they are decomposed into three independent source signals using the
ICA approach.
• Independent Component Analysis
Independent Component Analysis (ICA)is a method for Blind Source
Separation (BSS. BSS is a procedure involving the separation of a set of
independent signals from a set of mixed signals, where the mixing process
is unknown. To better understand in a better way the problem of signal
separation consider cocktail party example.
1.Cocktail Party Problem

The cocktail party problem is a classical example of BSS. Imagine you
are in a room with a large number of people talking simultaneously, like
in a cocktail party. Despite there are a large number of acoustic signals
in that room, in general people can follow properly one of the discussions.
This is because the human brain can handle the auditory source separation
problem, but in fact, it is a very tricky problem in digital signal processing.
Several approaches have been proposed for the solution of blind source
separation, some of the most successful are principal components analysis
(PCA) and ICA. The key for solving the problem is to have several indepen-
dent mixture signals, like in the case of the human ear, in which the mixture
of signals obtained from the right ear is different from the mixture of signals
obtained from the left ear.

2. Formal definition
If x = (x1 ; x2 ; ::: xm )T zero-mean m-dimensional random variables that
are, in fact, a linear mixture from s = (s1 ; s2 ; ::: sn )T n-dimensional inde-
pendent sources, the linear transformation of the source variables can be
expressed as:
x = As
where A is the m X n mixture matrix that contains the mixture coef-

ficients. It is usually assumed that the number of sources is equal to the
number of mixture signals, so m = n. In this context, both A and s are
unknown. The aim of ICA is to obtain a constant matrix W, which is the
inverse of the estimate of the matrix A, so that the linear transformation of
the observed variables become:
sb = W x
being sb an estimate of the vector s containing the source signals.
ICA finds the independent components by maximizing the statistical

independence of the source components. However, there exist many ways
to express independence. Motivated by the central limit theorem (CLT),
one way of measuring independence is the quantification of non-Gaussianity.
The CLT states that the sum of many independent random variables, each
with finite mean and variance, will tend to be distributed according to the
normal distribution. That means that the observed signals xi are more
gaussian than sources signals si . Therefore, separating the mixed signals is
understood as moving away from the normal distribution. Because of that,
one of the requirements of ICA algorithms is that sources signals cannot be
gaussian sources.
Mutual information is a mathematical tool used for measuring the non-
gaussianity of n random variables yi ; i = 1; :::; n, which measures, in fact,
the independence of that variables. The matrix W in Equation is then de-
termined so that the mutual information of the transformed components si
is minimized.
Though ICA is a robust mathematical tool, there are three ambiguities
should consider. The first one is that the variances of the independent
components cannot be determined, so the amplitude of the heart rate signal

recovered might not be the real signal amplitude. That is the reason why
some times the signal waveform is not recovered properly. Another ambiguity
is that the order in which ICA implies the signals is random. That involves
that algorithm for selecting the signal of interest among the three possible
signals must include. Also, the sign of the independent components is not
specified, which is not a big drawback for our purposes.
FastICA alsorithm is used for implementation
FastICA algorithm:
Following are the steps included in FastICA algorithm:-
1. Preprocess the data

Before the FastICA algorithm can be applied, the input vector data
should be centered and whitened.
2. Centering the data

The input data is centered by computing the mean of each compo-
nent of and subtracting that mean. This has the effect of making each
component have zero mean.
Thus:
x ←− x − E {x}
3. Whitened the data

Whitening the data involves linearly transforming the data so that
the new components are uncorrelated and have variance one. If is the
whitened data, then the co variance matrix of the whitened data is the
identity matrix:
n o
E x̃x̃t = I
This can be done using eigenvalue value decomposition of the co-

variance matrix of the data:
n o
E x̃x̃t = EDE T

where E is the matrix of eigenvectors and is the diagonal matrix of

eigenvalues. Once eigenvalue decomposition is done,the whitened data
is:
x ←− ED−1/2 E T x
4. Single component extraction

The iterative algorithm finds the direction fro the weight vector W
maximizing the non-Gaussianity of the projection W t X for the data
X. The function g(u)is the derivative of a non quadratic non linearity
function f(u). Hyvrinen states that good equations for (shown with
their derivatives and second derivatives)are:
0
f (u) = log cosh(u); g(u) = tanh(u); g (u) = 1 − tanh2 (u)
2 /2 2 /2 2 /2
f (u) = −e−u ; g(u) = ue−u ; g 0 (u) = (1 − u2 ) − e−u
1. Randaomise vector W.
n o n o
2. W + ←− E xg(wT ) − E g 0 (wT x) w
where E.... means averaging over all column-vectors of matrix x.
3. let w ←− w+ /kW + k
4. If not converged, go back to 2.
After then get the independent components and one of them is our
required signal without noise. This noise removed signal is used in next
module.
7.5.4 Parameter Extraction

1. Consider only green signal and discard red and blue signal
The frames are separated from input video. Then RGB i.e. red, green,
blue bands are separated. Green Band is used for Heart Rate(HR) and
Respiration rate(RR) as high absorption is observed at green band. So
consider only green band for further process. Red and green bands are used

for SpO2 estimation.
2. Select Region of Interest (ROI) on green signal

1. By Bucketing:
The initial green band image matrix is converted to 50 X 50 pixel matrix
by grouping 6X6 blocks of original matrix and stores its intensity average
into new matrix as one matrix as one pixel as shown in figure.
2. By removing high frequency noise:

It is observed that the pixels which are at the side of light source have
extreme high intensity value which causes error in final reading. The thresh-
old frequency is calculated by taking mean of intensities values and then
intensity values which are higher than this threshold is rounded at zero. By
this method obtain only required region of interest.
Figure 7.7: Bucketing Method
3. Obtain Inter Beat Interval (IBI)

After ROI selection mean intensities at green band is calculated. Then
the peaks are found from this signal and calculate average of mean of signal
and maximum intensity of signal. Then number of peaks which is greater
than this threshold is calculated.
How many beat interval into 60 second is calculate using following formula:
Heartrate = 60/mean(IBI)

Chapter 8
Software Testing
8.1 Introduction
Software testing is actually a set of different tasks whose primary purpose
is to fully exercise the computer based system. Although each test has a dif-
ferent purpose, all work to verify that the system elements have been properly
integrated and perform allocated tasks. Testing presents an interesting anomaly
for the software engineer. During earlier software engineering activities, the engi-
neer attempts to build software from an abstract concept to a tangible product.
Now comes testing. The engineer creates a series of test cases that are intended
to demolish the software that has been built. In fact, testing is the one step in
the software process that could be viewed as destructive rather than constructive.
Software engineers are by their nature constructive people.
Testing requires that the developer discard preconceived notions of the cor-
rectness of software just developed and overcome a conflict of interest that occurs
when errors are uncovered. If testing is conducted successfully (according to the
objectives stated previously), it will uncover errors in the software. As a secondary
benefit, testing demonstrates that software functions appear to be working accord-
ing to specification, that behavioral and performance requirements appear to have
been met. In addition, data collected as testing is conducted provide a good in-
dication of software reliability and some indication of software quality as a whole.
But testing cannot show the absence of errors and defects, it can show only that
software errors and defects are present. It is important to keep this (rather gloomy)
statement in mind as testing is being conducted.
51
Chapter 8. Software Testing
8.2 Test Cases

8.2.1 Definition
“A test case has components that describes an input, action or event and an
expected response, to determine if a feature of an application is working correctly.”
There are levels in which each test case falls in order to avoid duplication efforts.
• Level 1: In this level basic test cases written from the available specification
and user documentation.
• Level 2: This is the practical stage in which writing test cases depend on
actual functional and system flow of the application.
• Level 3: This is the stage in which some test cases and write a test proce-
dure. Test procedure is nothing but a group of small test cases maximum of
10.
• Level 4: Automation of the project this minimize human interaction with

system and thus QA can focus on current updated functionalities to test
rather than remaining busy with regression testing.
8.2.2 Fields in a Testcase

• Test cast ID
• Unit to Test : What to be verified
• Assumptions
• Test Data : Variables and their values
• Steps to be executed
• Expected Result
• Actual Results
• Pass/Fail
• Comments

8.2.3 Modules for Testing

• Capture Video and Frame Extraction Module
Test case No. Input Expected output Actual output Remark

1. Camera Capture Video Video Captured Pass
2. Video Frame Extraction Extracted Frames Pass
Table 8.1: Capture Video and Frames Extraction
• Obtain ROI And RGB Extraction from Frames

1. Frames Face and Forehead Region Pass
Forehead Region Captured
2. ROI RGB Extraction RGB Extracted Pass
Table 8.2: Obtain ROI And RGB Extraction from Frames
• Signal Separating using ICA and Signal Filtering using FFT

1. Mixture Separate RGB Separated Green Pass
of RGB Band Signals Band Signal
2. Green Filter Green Filtered Green Pass
Band signal Band Signal Band Signal
Table 8.3: Signal Separating using ICA and Signal Filtering using FFT
• Obtain Inter Beat Interval and Calculate Hear Rate

1. Filtered Green Values of Inter Array of Inter Pass
Band Signal Beat interval Beat interval
2. IBT Values Heart Rate Heart Rate (BPM) Pass
Table 8.4: Calculate Inter Beat Interval and Calculate Hear Rate

8.3 White Box Testing

White Box Testing is a software testing method in which the internal struc-
ture/ design/ implementation of the item being tested is known to the tester.
Sometimes called glass-box testing is a test case design method that uses the
control structure of the procedural design to derive test cases. Using white-box
testing methods, the software engineer can derive test cases that :
• Guarantee that all independent paths within a module have been exercised
at least once.
• Exercise all logical decisions on their true and false sides.
• Execute all loops at their boundaries and within their operational bounds
and
• Exercise internal data structures to ensure their validity.
White-box testing of software is predicated on close examination of proce-

dural detail. Providing test cases that exercise specific sets of conditions and/or
loops test logical paths through the software. The status of the program may be
examined at various points to determine if the expected or asserted status corre-
sponds to the actual status. Basis path testing is a white-box testing technique
first proposed by Tom McCabe. The basis path method enables the test case de-
signer to derive a logical complexity measure of a procedural design and use this
measure as a guide for defining a basis set of execution paths. Test cases derived
to exercise the basis set are guaranteed to execute every statement in the program
at least one time during testing.
In DTE Navigator one big obstacle when testing term extraction tools was
how to cross-reference the extracted terms with the entries. Obviously this has to
be done manually and it is extremely time-consuming. In some cases the extraction
process itself may have to do a data-validation rule in order to accept the data
and flow to the next phase.
8.4 Black Box Testing

Black Box Testing is a software testing method in which the internal struc-
ture/ design/ implementation of the item being tested is NOT known to the tester
also known as behavioural testing which focuses on the functional requirements

of the software. That is, black box testing enables the software engineer to derive
sets of input conditions that will fully exercise all functional requirements for a
program. Black box testing is not an alternative to white-box techniques. Rather,
it is a com- plementary approach that is likely to uncover a different class of er-
ror than white-box methods. When computer software is considered, black box
testing alludes to tests that are conducted at the software interface. Although
they are designed to uncover errors, black-box tests are used to demonstrate that
software functions are operational, that input is properly accepted and output is
correctly produced and that the integrity of external information is maintained. A
black-box test examines some fundamental aspect of a system with a little regard
for the internal logical structure of the software. Black- box testing attempts to
find errors in the following categories:
• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behaviour or performance errors
• Initialization and termination errors
By applying back-box techniques, derived set of test cases that satisfy the following
criteria:
• Test cases that reduce, by a count that is greater than one, the number of
additional test cases that must be designed to achieve reasonable testing.
• Test cases that tell us something about the presence or absence of classes of
errors, rather than an error associated only with the specific test at hand.
It is a software test design technique that involves identifying the developed

task (HTML links or queries) and effects (output is in the form of concepts, code
elemrnt and tasks), producing a Cause-Effect Graph, and generating test cases
accordingly. A limited number of important logical paths can be selected and
exercised. Important data structures can be probed for validity. The attributes
of both black and white box testing can be combined to provide an approach that
validates the software interface and selectively ensures that the internal workings
of the software are correct. Black box testing for this system was done to check
the internal testing i.e., the system is working properly in each module or not.

Proposed deductive program verification systems is to generate specifications

for given programs and then use these specifications as input for black-box testing
tools. In this way, black-box testing method can make use of information about the
program’s structure that is contained in the specification, and get a separation of
concerns and a clear interface between program analysis on the one hand and test-
case generation and execution on the other hand, which allows the combination
of a wide range of tools. The method for specification extraction using a program
verification calculus described in this DTE tool has been successfully implemented
in the Heart Rate Measurement system.

Chapter 9
Results and Discussion
9.1 Result snapshots

9.1.1 Graphical User Interface (JAVA And MATLAB):
The system final output is shown in the following figure. The options such as start
Camera,Select Video, process , HR value are available.
Figure 9.1: Graphical User Interface
57
Chapter 9. Results and Discussion
The face is detected from each frame and locate forehead region from captured
video. This forehead regions are stored to folder.
Figure 9.2: Frame Extracted Folder Window
Process option will process the captured video. All the steps in data processing
and parameter extraction module are executed and final value of heart rate are
estimated.
Figure 9.3: Graphical User Interface (JAVA and MATLAB)

9.1.2 Graphical User Interface (Python):

This system implemented in python, final GUI is shown below.In which forehead
region get located and process that video in real time for heart rate.
Figure 9.4: Graphical User Interface(PYTHON)
Figure 9.5: Graphical User Interface and Output Graph

9.2 Result Analysis

The simultaneous readings of heart rate are recorded for accuracy purpose
by using OlexPulse OLED Oximeter. We have recorded video using webcam using
java and processed in matlab. We get the readings by standard devices which will
be compared to final output by our system.
As per observation, it can be started as the data pre-processing with ICA gives
more accurate results. The ROI selection performed by removing high amplitude
signal gives better results.
Actual Reading Std Reading Accuracy Std Deviation
72 67 93.05556 3.535534
86 82 95.34884 2.828427
89 85 95.50562 2.828427
63 68 92.64706 3.535534
58 62 93.54839 2.828427
70 72 97.22222 1.414214
88 85 96.59091 2.121320
67 64 95.52239 2.121320
71 69 97.18310 1.414214
75 76 98.68421 0.707107
Average 95.59083 2.12132077
Co-relation Factor 0.903292505
Table 9.1: Comparison of actual result and Experimental Result
The table shows that the average accuracy in HR readings is 95.68253308%.

The correlation factor value is 0.903292505. i.e. it lies between 0.7 to 1.0 which
indicates that a srtong positive linear relationship via a firm linear rule.

9.3 Result Comparison Bar and Line Graph
Figure 9.6: Result Comparison Bar Graph
Figure 9.7: Result Comparison Line Graph

Chapter 10
Conclusion and Future Work
10.1 Conclusion
The system can monitor Heart Rate by using non-invasive and less costly
technique. No extra hardware is required. In this videos are recorded via a web-
cam, face detection algorithm is used to figure out the region and location. The
sequential face data are then applied to construct the normalized RGB raw data.
Since such raw data is not stationary, they should be set stationary by detrending;
otherwise, the spectrum of these raw data would only contain large energy at DC
component. The RGB data are then decomposed into three independent compo-
nents using ICA. As per study of previous work, the magnitude peak location in
the spectra of the three components is the corresponding BVP value in Hz. In
order to achieve an accurate estimation, several signal processing algorithms are
applied before the peak detection.
62
Chapter 10. Conclusion and Future Work
10.2 Future Work

This technology can be used for detection of Heart rate variability, Blood
volume loss, Arterial fibrillation and Cardiac autonomous function using simple
web-camera. The people are eating unhealthy food, lack of daily exercise, seden-
tary lifestyle and lack of proper knowledge, causes suffering from health disorders
like hypertension and diabetes. The available techniques are invasive as they re-
quired to take blood samples by pricking the skin. Some cases require the sugar
level checking after the regular time interval and this techniques are not afford-
able. If the system will be build for checking a glucose level from blood using this
techniques, it will place a milestone in the history of bio-medical engineering.

Bibliography
[1] S. Xu, L. Sun, and G. K. Rohde (2014) Robust eficient estimation of heart
rate pulse from video. Biomed. Opt. Express
[2] Y. P. Yu, B. H. Kwan, C. L. Lim, S. L. Wong, and P. Raveendran (2013)

Videobased heart rate measurement using short-time Fourier transform, in
Proceedings of International Symposium on Intelligent Signal Processing and
Communications Systems. IEEE
[3] Y. P. Yu, P. Raveendran, and C. L. Lim (2014) Heart rate estimation from
facial images using filter bank. IEEE International Symposium on Commu-
nications. Control and Signal Processing
[4] M.Z. Poh, D.J. McDuff, and R.W. Picard (2010) Noncontact, automated
cardiac pulse measurements using video imaging and blind source separation.
ASEE/IEEE Frontiers in Education Conference
[5] M.Z. Poh, D. McDuff, and R. Picard (2011) A medical mirror for non-contact
health monitoring. ACM SIGGRAPH 2011 Emerging Technologies
[6] H. Trotter (1959) An elementary proof of the central limit theorem. Math
[7] N. Tsumura, O. Nobutoshi, S. Kayoko, S. Mitsuhiro, S. Hideto, N. Hirohide,

A. Syuuichi, H. Kimihiko, and Y. Miyake (2003) Image-based skin color and
texture analysis/synthesis by extracting hemoglobin and melanin information
in the skin. ACM
[8] T. W. Lee, M. S. Lewicki, and T. J. Sejnowski(2004) ICA mixture models

for unsupervised classification of nongaussian classes and automatic context
switching in blind signal separation. IEEE Transaction
[9] S. Cook, M. Togni, M. C. Schaub, P. Wenaweser, and O. M. Hess (2006) High

heart rate: a cardiovascular riskfactor?.
64
BIBLIOGRAPHY
[10] L. Septiana, F. Hariyanto and K.P.Lin (2015) Comparison of Independent

Component Analysis (ICA) Algorithm for Heart Rate Measurement Based
on Facial Imaging. Springer
[11] Yuvraj D. Patil and Gyankamal J. Chhajed (2017) Survey on Non-Invasive

Methods of Heart Rate Measurement System. IRJET
[12] Shruti Madan Kshirsagar, Gyankamal J. Chhajed (2005) Heart Rate, Respi-
ration Rate and Oxygen Saturation Level Monitoring from Video Recording
by Smart phone camera. IJEDR
[13] T. Pursche, J. Krajewski, and R. Moeller (2012) Video-based heart rate mea-
surement from human faces. IEEE transaction
[14] S. Xu, L. Sun, and G. K. Rohde (2014) Robust eficient estimation of heart
rate pulse from video. Biomed. Opt. Express
[15] C. Takano and Y. Ohta(2007) Heart rate meaurement based on a timelapse

image. Med. Eng. Phys

Appendix A
Publications
Sr.No. Title of Paper Author(s) Name Conference/Journal Name Date Status

Survey on Non- Yuvraj Patil International Research Journal Volume 4,
1 Invasive of Heart and of Engineering and Technology Issue 5, Published
Rate Measurement Mrs. G. J. Chhajed (IRJET) May 2017
cPGCON-17, Computer
Heart Rate Mea- Engineering-PG Confer-
24-25th
surement System Yuvraj D. Patil ence,(Savitribai Phule Pune
2 March Presented
using Facial Video University), Organized by
2017
Processing Department of Computer
Engineering, SITRC, Nashik
Heart Rate Mea- Yuvraj D. Patil International Journal on Volume
surement System and Emerging Trends in Tech- 4,Special
3 Accepted
using Facial Video Gyankamal J. Chha- nology (IJETT) (cPGCON Issue, July
Processing jed Journal Publication) 2017
Table A.1: Publication on This Thesis
66
Chapter A. Publications
Figure A.1: Survey Paper Certificate

Figure A.2: cPGCON Certificate

Figure A.3: cPGCON Evaluation Sheet

Publications
Figure A.4: Hall Ticket


Heart Rate Measurement Using Facial Video

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Heart Rate Measurement Using Facial Video

Uploaded by

Copyright:

Available Formats

A

Heart Rate Measurement System using Facial Video

Submitted in Partial Fulfillment of the

Under the guidance of

Vidya Pratishthan’s Kamalnayan Bajaj Institute of Engineering

is a bonafide work carried out by his under the supervision of Mrs. G. J.

Mrs. G. J. Chhajed Prof. S. A. Shinde Prof. Mrs. S. S. Nandgaonkar

Dr. D. B. Hanchate Dr. M. G. Devamane

Internal Examiner External Examiner

Place: VPKBIET, Baramati. Signature of Guide

I take this opportunity to express my sincere thanks to my guide Mrs. G.

I wish to express my sincere thanks to Prof S. S. Nandgaonkar, (Head,

1. “Survey on Non-Invasive Methods of Heart Rate Measurement” published in

2. “Heart Rate Measurement System using Facial Video Processing” accepted in

3. “Heart Rate Measurement System using Facial Video Processing” presented in

Green band signals are extracted by applying Signal Separation algorithm

6 Project Estimate, Schedule 32

9 Results and Discussion 57

10 Conclusion and Future Work 62

1.1 Examples of ECG devices . . . . . . . . . . . . . . . . . . . . . . . 11

4.1 DFD Level-0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

6.1 Planner chart . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

7.1 Photoplethysmography imaging techniques . . . . . . . . . . . . . . 39

9.1 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . 57

A.1 Survey Paper Certificate . . . . . . . . . . . . . . . . . . . . . . . . 67

6.1 Project Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

8.1 Capture Video and Frames Extraction . . . . . . . . . . . . . . . . 53

9.1 Comparison of actual result and Experimental Result . . . . . . . . 60

A.1 Publication on This Thesis . . . . . . . . . . . . . . . . . . . . . . . 66

PPG Photoplenthesmography imaging technique

term monitoring of other physiological signals such as heart rate or respiratory

Heart Rate Measurement System 10 VPKBIET-Baramati

1.2 Existing Techniques

Figure 1.1: Examples of ECG devices

Heart Rate Measurement System 11 VPKBIET-Baramati

Figure 1.2: Examples of pulse oxiymetry devices

Heart Rate Measurement System 12 VPKBIET-Baramati

1.4 Flow Of Proposed System

Step 3: Parameter Extraction module extracts the values of heart rate.

1.5 Features of Proposed System

2. Independent Component Analysis algorithm is used as a for motion artifacts

3. Beat-beat interval computing algorithm, standard deviation, mean intensi-

1.6 Mathematical Model

Heart Rate Measurement System 13 VPKBIET-Baramati

Heart Rate Measurement System 14 VPKBIET-Baramati

2.1 Advancements in noncontact, multiparame-

1. Noninvasive method for detecting heart rate. It doesn’t require any

2.2 Video-based heart rate measurement from

2.3 Dynamic heart rate measurements from video

1. Minimum video processing required for heart rate measurement.

Heart Rate Measurement System 16 VPKBIET-Baramati

2.4 Comparison of Independent Component Anal-

Heart Rate Measurement System 17 VPKBIET-Baramati

2.5 A computationally efficient heart rate mea-

1. Minimum video processing required for heart rate measurement.

2.6 MA Reduction by using Traditional Approaches

Heart Rate Measurement System 18 VPKBIET-Baramati

2.7 MA reduction using Ring Sensor

1. Motion artifact reduction is handled.

2.8 Spatio-temporal filtering of thermal video se-

2.9 Mobile Cardiac Pulse Measurements

Heart Rate Measurement System 19 VPKBIET-Baramati