Professional Documents
Culture Documents
Final Year Project Report - Draft
Final Year Project Report - Draft
ABSTRACT
In the recent years the security has become a major concern. Gait recognition is a smart
surveillance system which is another form of providing security and identifying a person by
his walking style and foot pressure images. Foot plantar pressure is the pressure field that acts
between the foot and the support surface during everyday locomotor activities. Cumulative
foot pressure image is a 2-D data that records the spatial and temporal parameters during one
gait cycle. This attribute will allow us to identify a person in many scenarios including the
some scenes without monitor, jailhouse security system, entrance of houses it also helps us in
medical fields like physiotherapy. Information derived from such pressure measures is also
important in posture research for diagnosing lower limb problems, footwear design, sport
biomechanics, injury prevention and other applications.
Page | 1
Floor Sensor based Biometric for Video Surveillance System
CONTENTS
TOPIC PAGE NO.
Chapter 5: CONCLUSION
Chapter 6: BIBLIOGRAPHY
Page | 2
Floor Sensor based Biometric for Video Surveillance System
LIST OF FIGURES
Page | 3
Floor Sensor based Biometric for Video Surveillance System
CHAPTER 1
INTRODUCTION
1.1 Introduction
Identifying individuals using biometric methods has recently gained growing interest from
computer vision researchers for security purposes at places like airport, banks etc. Gait is one of
many physical and behavioral traits of an individual that can be used for recognizing a person.
Recognizing people using gait is an important area in biometrics with the applications in
homeland security, access control and human computer interactions. While most of the
researches for people identification using gait has been focused on computer vision based
techniques, there has been researches addressing gait recognition using foot pressure
information.
Gait refers to a person’s manner of walking. Gait recognition aims essentially to address
this problem by identifying people at a distance based on the way they walk and their foot
pressure images. A number of studies have established the potential of using gait information to
distinguish between individuals. Gait has even been used in criminal cases to identify
perpetrators based on their walking behavior. The ground reaction force and footprint are not as
popular as the limb movement does in pattern recognition and image process communities.
Ground reaction force is easily affected by the noise since it is a 1-D signal which records the
temporal information of gait. Recognizing footprint is intractable because it only contains the
rough foot pressure distribution.
In these cases, we consider that recognizing cumulative foot pressure image instead of
ground reaction force or footprint. Cumulative foot pressure image is a 2-D cumulative ground
reaction force during one gait cycle. Different from previous foot pressure data like ground
reaction force or footprint, it provides both temporal and spatial distribution information of the
gait. Different individual has his/her distinct gait, hence they have their own unique cumulative
foot pressure image. However, recognizing cumulative foot pressure image as a biometric is still
challenge since wearing different shoes bring the large inner-class appearance variation. In this
paper, we address the problem of developing a foot pressure image representation to achieve
tradeoff between discriminative power and translation-invariant ability for foot pressure image.
Gait recognition has several unique properties that other biometrics lack,
1. Human gait can be captured far away from the individual. In contrast, other biometric
systems [4-6] require that the individual is close to the biometric data collector or
physically contacts it.
2. Gait recognition can be done at low resolution, other biometric systems such as face
recognition may not work well.
4. Gait recognition can be done without the cooperation of the person. In contrast, other
biometrics such as fingerprints require the individual putting his/her finger on the sensor
for data acquisition.
5. Gait features are hard to be impersonated. This is because gait recognition typically
uses human silhouettes and activities. This property is very important to crime analysis.
Machine learning is the scientific field dealing with the ways in which machines learn
from experience. For many scientists, the term “machine learning” is identical to the term
“artificial intelligence”, given that the possibility of learning is the main characteristic of an
entity called intelligent in the broadest sense of the word. Various popular feature learning
methods such as sparse coding, Robust PCA, CCA, and Group sparse coding have been explored
in multi-view multi-modality gait recognition and intelligent visual surveillance system
applications.
Gait recognition is a popular pattern recognition problem for which attracts a lot of
researchers from different communities such as computer vision, machine learning, biomedical,
forensic studying and robotics. This problem also has great potential in industries such as visual
surveillance. Hence, The Intelligent Recognition & Digital Security Group, which was formed in
1998 by Prof. Tieniu Tan at NLPR (National Laboratory of Pattern Recognition), developed this
CASIA dataset since Dec. 10, 2001.
This CASIA Gait Recognition Dataset current contains 4 subset: Dataset A(standard dataset),
Dataset B (multi-view gait dataset) and Dataset C (infrared gait dataset), and Dataset D(gait and its
corresponding footprint dataset). The contributions of the project are two folds. 1) We create a new
database contains gait and foot pressure image (footprint), which would help further studies on gait. It
becomes Gait CASIA Part D sub dataset.
CHAPTER 2
LITERATURE SURVEY
Early medical and psychological studies [1][2] showed that human gait had 24 different
components, which could be used for identifying an individual. It was shown that the light points
attached to individuals’ joints could be used for representing a human motion, and point light
displays could be used for distinguishing human activities [3]. The above works indicates that
every individual had unique muscular-skeletal structure which could be used for identifying
him/her. Therefore, gait recognition is feasible.
This survey is an attempt to bring a comprehensive, most up-to-date research and analysis of
advances and the state of practice in gait-based authentication.
Literature surveys of some papers-
In 1994, the earliest gait recognition system was proposed by authors in Reference [7],
which based on a small gait database. Then, the Defense Advanced Research Projects
Agency (DARP) developed the famous Human ID program, which set up the first
publicly accessible database for gait recognition. Since then, researchers have done many
works on gait recognition. Recently, gait recognition has been done through sensor
readings such as accelerometer, floor sensors etc,.
Nakajima et al. used floor sensors for collecting gait data, such as pressure, for the first
time. Finally, in 2005, Otero first introduced the continuous wave radar for collecting gait
data.
As described in the paper that was presented at the NISk-2010 conference, Apart from
the machine vision (MV) based and floor sensor (FS) based gait recognition, the wearable
sensor based gait approach is the newest. This is based on attaching or wearing motion
recording sensors on the body of the person in different places; on the waist, pockets,
shoes and so forth. The wearable sensors (WS) can have several purposes due to
retrieving numerous types of data. Sensors of different types can for instance be
CS&E, BIET Davangere Page 1
Floor Sensor based Biometric for Video Surveillance System
After modeling the whole human body, model-based feature representation methods
typically use distances or joint angles on human body for gait recognition. For example,
Bobick and Johnson modeled the human body using four distances, namely left-right-foot,
head-pelvis, head-foot, and foot-pelvis distances. Wang et al. divided the human body into
14 parts and used joint angle trajectories in each part for identifying individuals. They also
combined Procrustes shape analysis for improving recognition rate.
Hough transform (VHT) for extracting features from legs and motions
In 2004, Jung et al suggested combining static and dynamic foot pressure features to identify
walking individuals. They used a mat containing 40 × 80 pressure sensors (1 × 1 cm2 resolution)
to record footprints and COP trajectories and collected one-step footprints from 11 participants.
Forty footprints were recorded from each subject for two months. An overall good classification
accuracy was obtained using the hidden Markov model and Levenberg–Marquart learning
methods. One strength of the study was that data were recorded over a long period of time and
tended to demonstrate that foot pressure features are time invariant to a substantial degree. The
study’s limitations included the small sample size and the fact that the subjects had to walk
barefoot, making the method difficult to apply in practice.
In an article published in 2007, Suutala and Röning described a method for using a pressure-
sensitive floor to identify individuals. They covered the floor of their research laboratory with
100 m2 of an electro-mechanical film sensitive to pressure and collected gait pressure data from
10 individuals. The researchers focused on the pressure intensity profiles, rather than on
pressure trajectories or foot shape. They applied several classifying algorithms, of which support
vector machines and multilayer perceptron were the most accurate. An above average
classification accuracy was achieved when multiple features collected from several consecutive
steps were combined. Here the main limitation was the small database. .
In the 2012 study, Pataky et al analyzed dynamic foot pressure patterns through plantar pressure
imaging using a pedography platform that recorded footprints with 5 mm resolution. They
recruited 104 individuals and collected 1040 barefoot steps. Several pre-features characterizing
the pressure patterns where extracted, and a dimensionality reduction technique was used. A
one-fold nearest-neighbor classifier was applied with cross-validation. The results show that the
best feature was the pressure–time integral (PTI), with a classification rate of 99.6%. Overall,
this study demonstrated that plantar pressure patterns are highly unique among individuals. The
main practical limitation was that they investigated unshod walking.
(such as a key or card). In the recent years, biometric methods for identifying the individual is
gaining interests from researchers. As fingerprints, face and eyes are currently in use for the
biometric method, many researchers also found that foot can also have the potential to be
considered as a biometric factor in identifying individuals with the help of foot pressure images.
Many approaches have done in this field but the small data size, low accuracy levels and
other constraints made them not suitable for practical implementations.
METHODOLOGY
3.1 Methodologies:
Data
Acquisition
Applying
Pre- Segmenta CFPI Predicted
SVM
processing tion Feature Output
Database Algorithm
Unlabeled
Unlabeled -Gray Scale -Contour Extraction
CASIA-D -Thresholding
CASIA-D Detection
Dataset
Dataset
PHASE 1
Data acquisition is the process of sampling signals that measure real world physical
conditions and converting the resulting samples into digital numeric values that can be
We are working on the standard data set i.e. CASIA-D dataset that consist of 88 subjects.
When a subject walked in the scene, he/she was asked to walk naturally along a straight
line through the pressure sensor 5 times first and 5x88 normal walking cumulative foot pressure
images were captured for each subject. There are 20 female subjects, 66 male subjects. All
subjects are Asian aged between 20 and 60. The data are representative and ensure the future
experiments to be not limited in any specific group of people. After normal walking, the subjects
were asked to walk fast through the pressure sensor 5 times. In that case, there are totally
10x88=880 cumulative foot pressure records, three cumulative foot pressure images for each
record, so there are totally 2640 images for considering the effect of walking speed.
PHASE 2
3.1.2 Pre-processing:
grayscale image is one in which the value of each pixel is a single sample representing only an amount of light.
Average method
Average method - is the simplest one, in which we have to just take average of 3 colours channel.
Since it is an RGB image, so it means that you have add r with g with b and then divide it by 3 to
get your desired grayscale image.
Grayscale = (R + G + B / 3)
But using this method, we will get more of a blackish image rather than a grayscale image. This
happens because in an image red color has more wavelength than the other colors.
Weighted method or luminosity method – this method solves the problem that was occurring due
to more wavelength of the red colour in RGB channel. So, now we will reduce the weightage of
red colour in the image.
where x, y, z are ratio in which we want these color to be mixed for our image to represent them
as gray scale image.
3.1.3 Thresholding –
Here, the matter is straight forward. If pixel value is greater than a thresholding, it is
assigned one value (may be white), else it is assigned another value (may be black).
then f (x, y) = 0
else
f (x, y) = 255
Where
T = Threshold Value
Parameters:
cv2.THRESH_BINARY:If pixel intensity is greater than the set threshold, value set to
255, else set to 0 (black).
cv2.THRESH_BINARY_INV: Inverted or opposite case of cv2.THRESH_BINARY.
cv.THRESH_TRUNC: If pixel intensity value is greater than threshold, it is truncated to
the threshold. The pixel values are set to be the same as the threshold. All other values
remain the same.
cv.THRESH_TOZERO: Pixel intensity is set to 0, for all the pixels intensity, less than the
threshold value.
The simplest way of representing Cumulative foot pressure images data is dimensions of
the foot like area, perimeter, length etc. However, since many individuals may have the same
dimensions, the recognition rate of this feature representation may be low. But, in a published
short talk these features were a part of the feature set.
Therefore, dimensional properties are a weak biometric. We can also have features based on body mass
but, body mass can be easily computed from pressure of the floor sensor as M = kF, in which F
represents the pressure collected by the floor sensor, k represents the coefficient, and M represents the
body mass. More features can be extracted from floor-sensor data (pressure), such as cadence and
length of stride. Middleton et al. placed 1, 536 sensors in an area of 3cm2.
Stride length and stride cadence are used in video-based gait recognition systems too.
For finding these features we used contours that we got from the segmentation. In
OpenCV these contours can be used to find the different properties from an image like length,
area, arc length of the images, moments or can be used to fit shapes around the objects.
PHASE 4
3.1.5 Applying Support Vector Machine (SVM) Classification Algorithm-
The Support Vector Machine (SVM) Classification is similar to the SVR. In SVM, the line that is
used to separate the classes is referred to as hyperplane. The data points on either side of the hyperplane
that are closest to the hyperplane are called Support Vectors which is used to plot the boundary line.
In SVM Classification, the data can be either linear or non-linear. There are different kernels that
can be set in an SVM Classifier. For a linear dataset, we can set the kernel as ‘linear’.
On the other hand, for a non-linear dataset, there are two kernels, namely ‘rbf’ and ‘polynomial’.
In this, the data is mapped to a higher dimension which makes it easier to draw the hyperplane.
From the above diagram, we can see that there are two classes of shapes, rectangle and circle. As
it is difficult to draw a SVM line in the 2D Plane, we map the data points to a higher dimension (3D
Plane) and then draw the hyperplane. It is then brought down to the original plane with the SVM
In this way, the SVM Classifier can be used to classify a data point to which class it belongs from
Mathematical Analysis:
A point in a space:
When we define x in a real space, we understand its domain, and on mapping a function for y = f(x) ,
we get range and co-domain.
So, initially we are given the data which is to be separated by the algorithm.
The data given for separating / classifying is represented as a unique point in a space where each point
is represented by some feature vector x.
We are applying a similar concept of domain, range, mapping a function for the data points here ,
instead of real space, we have a vector space for x.
Decision Boundary:
Equation of Hyperplane :
The equation of the main separator line is called as a hyperplane equation.
Let us look at the equation for a straight line with slope m and intercept c .
The equation becomes : mx + c = 0
(To notice : we have fit a straight / linear line which is 1-D in a 2-D space)
The hyperplane equation dividing the points (for classifying) can now easily be written as:
H: wT(x) + b = 0
Distance Measure:
The distance of any line, ax + by + c = 0 from a given point say, (x0 , y0) is given by d.
Similarly, the distance of a hyperplane equation: wTΦ(x) + b = 0 from a given point vector Φ(x0) can be
easily written as :
Because of the advantages of SVM over other classifier algorithms, we use it to train model with the
dataset (csv file), which we obtained from the feature extraction. Using this model, we can predict the
individual/ person by giving the features as input.
Minimum: 1 GB of RAM
Minimum: 10 GB of Hard disk space
OS: Windows 7 or Higher Versions
Scikit-image processing
Pandas
Numpy
Open-CV
CONCLUSION
BIBILOGRAPHY
[12] Miler Jerkovic Vera, Bojanic Dubravka, Jorgovanovic Nikola, Ilic Vojin, and Petrovacki Balj
Bojana. 2013. Detecting and removing outlier(s) in electromyographic gait-related patterns. Journal
of Applied Statistics 40, 6, 1319–1332.