PhD Progress

on
Design and Implementation of Novel Technique for
Multi View Face Detection System

Shivkaran Ravidas
13/PhD/Engg./013

Under the Guidance of

  Dr. M. A. Ansari

School of Engineering
Gautam Buddha University
School of Engineering
1
 Contents

Introduction
Literature Review
Summary of Research work
Research objectives
Work done in last semester
Results
Conclusion
Reference

School of Engineering
2
Introduction
Face Detection
− Given an arbitrary image the goal of a face detection is to
determining whether or not there are any faces on the image
and, if present, return the image location and extent of each
face.

Extent of face

Location of face

School of Engineering
3
 Importance of Face Detection
The first step for many automatic face recognition
system system.
First step in many Human Computer Interaction
systems.
Expression Recognition
Cognitive State/Emotional State Recognition.
First step in many surveillance systems.
Tracking: Face is a highly non rigid object.
Video coding.
And many more…….

School of Engineering
4
Face Detection: Challenges
Out-of-Plane Rotation: frontal, 45 degree, profile, upside
down .
Presence of beard, mustache, glasses etc.
Facial Expressions.
Occlusions by long hair, hand etc.
In-Plane Rotation.
Image conditions:
 Size
 Lighting condition
 Distortion
 Noise
 Compression

School of Engineering
5
 Literature Review
S. Author’s name Findings Title Journal’s name
No.  
1. Farfade, Sachin Proposes Deep Dense Face Detector Multi-view Face Detection Proceedings of the 5th ACM
Sudhakar, (DDFD), a method that does not Using Deep Convolutional on International Conference
require pose/landmark. Neural Networks on Multimedia Retrieval.
ACM, 2015
2. They propose a cascade architecture A Convolutional Neural Proceedings of the IEEE
Li, Haoxiang, et built on convolutional neural Network Cascade for Face Conference on Computer
al networks (CNNs) with very powerful Detection Vision and Pattern
discriminative capability, while Recognition. 2015.
maintaining high performance.

3. M. Pamplona It describe a real-time 3D face Orthogonal Projection Accepted Manuscript in

Segundo, L. detector based on boosted cascade Images for 3D Face Detection Pattern Recognition Letters,
Silva, O.R.P. classifiers that uses a scale-invariant S0167-8655(13)00369-3, 2013
Bellon, S. Sarkar image representation to improve
efficiency.

4. Computer Vision and Pattern

Provides a unified model for face Face Detection, Pose Recognition (CVPR), 2012
Xiangxin Zhu et detection, pose estimation, and Estimation, and Landmark IEEE Conference on , vol.,
al landmark estimation in real-world, Localization in the Wild
cluttered images. no., pp.2879,2886, 16-21 June
2012

5. Bongjin Jun, This paper proposes a novel face Robust face detection using pattern recognition, Volume
Daijin Kim detection method using local gradient local gradient patterns and 45, Issue 9, September 2012
patterns (LGP). evidence accumulation

School of Engineering
6
 Literature Review (Cont..)
S. Author’s name Findings Title Journal’s name
No.  
6. They propose a cascade architecture A Convolutional Neural Proceedings of the IEEE
Li, Haoxiang, et built on convolutional neural Network Cascade for Face Conference on Computer
al networks (CNNs) with very powerful Detection Vision and Pattern
discriminative capability, while Recognition. 2015.
maintaining high performance

7. It extended the viola jons principle for Aggregate Channel Features Biometrics (IJCB), 2014 IEEE
Yang, Bin, et al the multi-view face detector using for Multi-view Face Detection International Joint
aggregate channel features Conference on. IEEE, 2014.

8. Chen, Dong, et It present a new state-of-the-art Joint cascade face detection Computer Vision–ECCV
al approach for face detection. The key and alignment 2014. Springer International
idea is to combine face alignment Publishing, 2014.
with detection

9. Liao, Wenjuan, et As an important supplement to wired Wireless Video Surveillance MultiMedia Modeling.
al video in video surveillance System Based on Incremental Springer International
applications, wireless video has taken Learning Face Detection Publishing, 2015
increasing attentions and has been
extensively applied into projects like
“Safe City”.

10. Jiang, Wenbo, This paper is used to solve the time- Detection Based on Feature Web of Conferences. Vol. 22.
and Xiaoming consuming problem of training Tailoring and Skin Color Space EDP Sciences, 2015
Xie samples in Adaboost algorithm and
propose an improved FTAdaboost
algorithm based on feature tailoring

School of Engineering
7
Research Objectives
 To do exhaustive literature Survey in Multi-View face detection and its
applications in health monitoring (Expression etc.)

 To find out optimum algorithms for the Multi-View Face Detection.

 To find out parallel architectures for multi-view face detection.

 To analyze multi-view face detection techniques and to find out

optimum algorithm that can be utilized in health monitoring and
biometrics applications.

 Comparative performance analysis of multi-view face detection

techniques in health monitoring systems.

School of Engineering
8
 Summary of Research Work
Completed Course work .
Completed Pre-RDC , RDC and approved thesis Title.
Exhaustively Literature Survey has been done in Related Area. (Around 80 papers)
Designed and implemented System for Multi view Face detection using
Deep Convolutional Neural Network
Multi-View Face Detection: A Comprehensive Survey.
(International Journal of Computer Science and Mobile Computing, Vol.3 Issue.3, March- 2014, pg.
1193-1203)
The scope and challenges in Multi-View face detection. (ISIAM
conference, Sharda University)
Work is going on to fulfill the following objectives:
! To find optimized technique for multi view face detection that can
be used for health monitoring system.
! Comparative performance analysis of multi-view face detection
techniques health care.

School of Engineering
9
 WORK DONE IN LAST SEMESTER
1. Communicated in a Journal Computer Science and Information Systems
(SCI Expanded) titled as “An Efficient Scheme of Deep Convolution
Neural Network for Multi View Face Detection .

2. Communicated paper in International conference on Nanotechnology for

Instrumentation and Measurement Workshop , NANOfIM2017 titled as :
“Pose Invariant Face Detection using Deep Convolutional Neural
Network and Calibrated CNN Structure”.

School of Engineering 10
 WORK DONE IN LAST SEMESTER
We have developed a algorithm for multi-view face detection using deep
convolution neural network. The steps of implementation are described
below:

Step 1: In the implementation, detection of face and retrieval of image will

be attained with the help of direct visual matching technology which match
the face directly. This technology makes use of similarity metrics of an image
which can be normalized correlation.
 
The similarity between the two images is measured through similarity
measure, denoted by S (Ia, Ib) ,Where, Ia and Ib are the two images between
which the similarity is being measured.
 
Step 2: The next step is measuring probabilistic similarity or ∆(the measure
of intensity difference between the two images) given by Probabilistic
similarity .

School of Engineering 11
 DESIGN OF DEEP CONVOLUTIONAL
CNN
We have developed a algorithm for multi-view face detection using deep
convolution neural network. The steps of implementation are described
below:

Step 1: In the implementation, detection of face and retrieval of image will

be attained with the help of direct visual matching technology which match
the face directly. This technology makes use of similarity metrics of an image
which can be normalized correlation.
 
The similarity between the two images is measured through similarity
measure, denoted by S (Ia, Ib) ,Where, Ia and Ib are the two images between
which the similarity is being measured.
 
Step 2: The next step is measuring probabilistic similarity or ∆(the measure
of intensity difference between the two images) given by Probabilistic
similarity .

School of Engineering 12
DESIGN OF DEEP CONVOLUTIONAL CNN
(Cont.)

.
 
 
 Step 3: Further a neural network will be developed and trained in order to
enhance the outcome from this analysis.
 
Step 5: Next to that, training and verification will be adopted to test other
images which involve similar face features.

School of Engineering 13
FLOWCHART FOR TRAINING
Start

Initialize training
Epoch=1

Initialize weights
and Biases

Present input image and

calculate output values

Calculate RMSE

Epoch= Epoch +1
Yes
RMSE ≤ RMSEmin

No
Yes
RMSE ≥ RMSE max Stop Training

No
Update weights and Bias

School of Engineering 14
Convolutional Neural Networks
C layers are
convolutions, S
layers
pool/sample

Often starts with

fairly raw features
at initial input
and lets CNN
discover improved
feature layer for
final supervised
learner

15
School of Engineering 15
Deep Learning Overview
Train networks with many layers.
Multiple layers work to build an improved feature space
 First layer learns 1st order features (e.g. edges…)
 2nd layer learns higher order features (combinations of first
layer features, combinations of edges, etc.)
 In current models layers often learn in an unsupervised
mode and discover general features of the input space –
serving multiple tasks related to the unsupervised
instances (image detection , recognition, etc.)
 Then final layer features are fed into supervised layer(s)

School of Engineering 16
Sub-Sampling (Pooling)
Convolution and sub-sampling layers are interleaved
Sub-sampling (Pooling) allows number of features to be diminished, non-
overlapped.
Reduces spatial resolution and thus naturally decreases importance of exactly
where a feature was found, just keeping the rough location
Averaging or Max-Pooling (Just as long as the feature is there, take the max,
as exact position is not that critical)
2x2 pooling would do 4:1 compression, 3x3 9:1, etc.
Pooling smooths the data and makes the data invariant to small translational
changes
Since after first layer, there are always multiple feature maps to connect to
the next layer, it is a pre-made human decision as to which previous maps the
current map receives inputs from

CS 678 – Deep Learning 17

School of Engineering 17
RESULTS

Fig 3. Face detected Images

School of Engineering 18
RESULTS (CONT.)

Fig4. Pose invariant Face detected Images

School of Engineering 19
RESULTS (CONT.)

Fig.9 Results showing Misclassifications

School of Engineering 20
RESULTS (CONT.)
Detection Rate with multi resolution
1.2
Detection Rate 1

0.8

0.6

0.4

0.2

0
0 2000 4000 6000 8000 10000 12000

Number of False Detections

Detection Rate without multi resolution

1.2
1
0.8
Detection
Rate

0.6
0.4
0.2
0
0 2000 4000 6000 8000 10000 12000 14000

Number of false Detections

School of Engineering 21
RESULTS

Fig 1. Pose invariant Face detected Images Fig 2 DCNN training Progress

School of Engineering 22
 DETECTION RESULTS FOR DIFFERENT CNN
STRUCTURE
(a) Input/Test Image

(b) Image after 12-net CNN © Image after 24-net CNN (d) Image after 48-net CNN

(e) Output face detected Image

School of Engineering 23
 Work to be done
Work is going on to fulfill the following objectives:

To find optimized technique for multi view face detection

that can be used for health monitoring system

Comparison of various multi view techniques suitable for

health monitory systems.

School of Engineering
24
 References
1 Li, Haoxiang, et al. "A Convolutional Neural Network Cascade for Face Detection." Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. 2015

2 Segundo, M. Pamplona, et al. "Orthogonal projection images for 3D face detection." Pattern Recognition Letters 50 ,
2014

3 Li, Haoxiang, et al. "A Convolutional Neural Network Cascade for Face Detection." Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition. 2015..

4 Yang, Bin, et al. "Aggregate channel features for multi-view face detection."Biometrics (IJCB), 2014 IEEE International
Joint Conference on. IEEE, 2014

5 Jiang, Wenbo, and Xiaoming Xie. "Face Detection Based on Feature Tailoring and Skin Color Space." MATEC Web of
Conferences. Vol. 22. EDP Sciences, 2015

6 Bongjin Jun, Daijin Kim ,”Robust face detection using local gradient patterns and evidence accumulation,” pattern
recognition, Volume 45, Issue 9, September 2012

7 Liao, Wenjuan, et al. "Wireless Video Surveillance System Based on Incremental Learning Face Detection." MultiMedia
Modeling. Springer International Publishing, 2015

8 Jones, M.; Viola, P.,”Fast Multi-view Face Detection”, MITSUBISHI ELECTRIC RESEARCH LABORATORIES, TR2003-
96 August 2003

School of Engineering
25
 References (Cont.)
9 A. Pentland, B. Moghaddam, and T. Starner. "View-based and Modular Eigenspaces of Face
Recognition". Proc. of IEEE Computer Soc. Conf. on Computer Vision and Pattern
Recognition, pp. 84-91,.Seattle, Washington. June 1994

10 H. A. Rowley, S. Baluja, and T. Kanade. "Neural network-based face detection". IEEE

Transactions on Pattern Analysis and Machine Intelligence 20 ,1998.

11 [M. Pamplona Segundo, L. Silva, O.R.P. Bellon, S. Sarkar,” Orthogonal Projection Images
for 3D Face Detection”, Accepted Manuscript in Pattern Recognition Letters, S0167-
8655(13)00369-3, 2013

12 H. Schneiderman and T. Kanade. "A Statistical Method for 3D Object Detection Applied to
Faces and Cars". Proc. IEEE Computer Soc. Conf. on Computer Vision and Pattern
Recognition, 2000

13 P. Viola and M. Jones. "Robust real time object detection". IEEE ICCV Workshop on
Statistical and Computational Theories of Vision, Vancouver, Canada, July 13, 2001.

14 S. Z. Li, et al. "Statistical Learning of Multi-View Face Detection". Proc. of the 7th European
Conf .on Computer Vision. Copenhagen, Denmark. May, 2002.

15 Ching-chun Huang , Jay Chou , Jia-hau Shiu and Sheng-Jyh Wang ,”Multi-view Face
Detection Based on Position Estimation over Multi-camera Surveillance System

26
 Thank You…

School of Engineering
Structures of CNN

Fig 1: 12-net CNN

Fig 2: 24-net CNN

School of Engineering
School of Engineering
Face Detection in expression Recognition
 The face is one of the most powerful channels of nonverbal
communication.
 Expressions can be classified in terms of facial actions that
cause an expression or “typical” emotions
 Facial muscle activity can be described by a set of codes
◦ Visual Facial changes can be detected for Health Monitoring system like
ICU

School of Engineering
30
Health Monitoring by Facial Expressions

Happy
Sad
.
. Afraid
.
Face Detection Angry
PCA/LBP/NN
(Localization)
Surprised
Distress
.
.
.

School of Engineering
31