SRM INSTITUTE OF SCIENCE AND TECHNOLOGY

SCHOOL OF COMPUTING
DEPARTMENT OF COMPUTING TECHNOLOGIES
18CSP107L - MINOR PROJECT

Real-Time Hand Gesture Recognition

System
Batch ID: B185

Student 1 Reg. No: RA1911003010624

Guide name: Mr. S Saminathan
Student 1 Name: Pratik Agrawal
Designation: Assistant Professor
Student 2 Reg. No: RA1911003010616
Department: CTECH
Student 2 Name: Shaswat Gupta
1
 Index
S.No. Content Slide No.

1 Abstract 3

2 Introduction 4

3 Motivation 5

4 Innovation idea of the project 6

5 Requirement Gathering 7

6 Literature Survey 8

7 Inference 13

8 References 14

2
 Abstract
● The goal of this research is to present and develop a technique for hand gesture
recognition.

● This project and experiment will be conducted with the aim of utilizing the human
hands as an object to operate computers. It is intended to support and use
technologies in the field of contactless shopping/payments.

● In order to use this program, the person needs to be in front of a computer webcam.
The webcam will be used to recognize the shape and the pattern of the presenters’
hands.

● The result of this project is a program that can be used for improve the user
experience of contactless systems and make safer transactions in a time of global
pandemic, where social distancing is one of the main things to consider.

3
 Introduction
● The vision-based technology of hand gesture recognition is an important part of
human-computer interaction (HCI). Technologies such as speech recognition and
gesture recognition receive great attention in the field of HCI.

● Gesture is a symbol of physical behavior or emotional expression of a person.

Gestures can be 2 types - body gesture and hand gesture. Furthermore, the gestures
can be static and dynamic. For the latter, each movement represents some meaning.

● Humans use their fingers hand gestures more often to point to something, to count,
to express or underline some point.

● Contactless interaction process with computers are becoming a must-have and

desired feature these days of global pandemic of 2020. Contactless payments for
services, especially the ones without cash i.e. digital payment services may boost
during this period worldwide.

4
 Existing Techniques
● Wired Gloves

● Depth-aware Cameras

● Stereo Cameras

● Controller-based Gestures

5
 Literature Review
Tracker Based Analysis –

● Glove based Analysis – Attach a sensor with gloves mechanical or optical to transduces
flexion of fingers into electrical signals for hand posture determination and additional
sensor for position of the hand

● Advantages - Direct measurement of hand and finger parameters, provision of data, high
sampling frequency, easy if use, line of sight, low cost version and translation
independency feature of data.

● Disadvantages - Difficulty in calibration, reduction in range of motion and comfort,

noise in inexpensive system, expensiveness of accurate system. Moreover it’s
compulsory for user to wear cumbersome device.
.

6
 Literature Review
Tracker Based Analysis –

● Body Suit– Process of small place of strategically dots placed on human body, people
can perceive patterns such as gestures, activities, identities and other aspects of body.
One way of approach is recognition of postures and human movements is optically
measure of 3D position such as markers attached to body and then recovers time varying
articulate structure of body.

● Advantages - At high sampling rate it provides reliable results.

● Disadvantages - They are cumbersome and very expensive. Non-trivial calibration.

Several cameras used by optical system which is typically offline data process, lack of
wires and tether is major disadvantages.

7
 Literature Review
Vision Based Analysis -
● In this approach human beings get information from their surroundings, and this is
probably most difficult approach to employ in satisfactory way. Many different
implementations have been tested so far.
● Several cameras attached to this model to determine parameters corresponding for
matching images of the hand, palm orientation and joint angles to perform hand gesture
classification.
● Vision-based interfaces use one or more cameras to capture images and interpret those
images to produce visual features that can be used to interpret human activity and
recognize gestures.
● This technique was also used by us for recognizing hand gestures in real time. With the
help of a web camera, we took pictures of hand on a prescribed background and then
applied the classification algorithm for recognition.

8
 Literature Review
Ref. Scope of analysis
Lalit K. Phadtare A survey of hand gesture modelling,
analysis, and recognition.
Anushree Pillai A survey of vision-based static hand
posture and temporal gesture recognition
approaches and their applications.
Moeslund and A survey of vision-based human motion
Granum capturing methods focusing on
initialization, tracking, pose estimation,
and recognition.
D.K. Vishwakarma A review of vision-based hand gestures
on different aspects like feature set,
classification methods and gesture
representation.
Contributions
3D hand models better describe and
discriminate gestures but with higher
computational complexity. Appearance-based
models are limited in gesture description
despite being simpler to implement.
An investigation of feature selection, data
collection for visual gesture learning, various
algorithms for static and DG recognition and
their variants in sign language recognition.
The lack of an alphabet vocabulary of motion
data for training, the large amount of time
required to capture and label human motion
data, the problem of incremental updates,
motion ambiguity due to movement projection
on planes.
Analysis of gesture-based HCI under ambient
lighting conditions and high contrast stationary
backgrounds.
9
 Literature Review
Ref. Scope of analysis
K.Peterson A review of advances in video-based
human capture and analysis and open
problems in the automatic visual
analysis of human movement.
C. Patlolla A review of hand pose estimation
and method and capturing of the real 3D
S. Mahotra  hand motion in HCI.
Mitra and A survey of hand gesture and facial
Acharya expression recognition methods with a
discussion on different tools for gesture
recognition.
Moni and Ali A literature review of different
approaches to vision-based gesture
recognition using HMMs.
Contributions
Investigated automatic initialization of model shape, appearance,
and pose; reliable detection and tracking of multiple people in
unstructured outdoor scenes with partial occlusion; human
motion reconstruction from multiple views and from monocular
videos.
Comparison of methods on the effective number of DoF, number
and types of cameras, system's operation in a cluttered
background, use of hand model constraints and observable pose
restrictions. Discussion of problems due to lack of ground-truth
data, high-dimensionality, self-occlusions, uncontrolled
environments, and rapid hand motion.
Discussion on HMMs, particle filtering, and condensation
algorithm, FSMs, ANNs, Gabor filtering and optical flow;
hybridisation of HMMs and FSMs; different facial expression
modelling approaches; use of soft computing tools.
Difficulty in achieving a continuous online recognition with
minimal time delay because the starting and ending points of a
gesture are not known. Problems due to assumptions about
lighting, background, and signer's location are discussed.
10
Flow Chart

11
 BRIEF OUTLINE OF THE
IMPLEMENTED SYSTEM
Hand Gesture module can be divided into three parts-

● Preprocessing - The preprocessing prepares the image sequence for the recognition, so
before calculating the diagonal Sum and other algorithms, a pre-processing step is
performed to get the appropriate image, which is required for real time classification.

• Skin Modeling
• Removal of Background
• Conversion from RGB to binary
• Hand Detection

12
 BRIEF OUTLINE OF THE
IMPLEMENTED SYSTEM
Hand Gesture module can be divided into three parts-

● Feature extraction of processed image - There are four types of algorithms that we
studied and implemented namely as followings:
• Row vector algorithm
• Edging and row vector passing
• Mean and standard deviation of edged image
• Diagonal sum algorithm

● Real time classification - In real time classification the system developed tries to
classify the gestures not saved before but given at the run time. The system first trains
itself with the user count gestures at the run time and then tries to classify the newly
given test gestures by the user. The algorithm used by the system for real time
classification is the diagonal sum algorithm.

13
Code

14
 References

• J.Jenkinwinston (96207106036), M.Maria Gnanam (96207106056), R.Ramasamy

(96207106306), Anna University of Technology, Tirunelveli: Hand Gesture
Recognitionsystem Using Haar Wavelet.
• Laura Dipietro, Angelo M. Sabatini, Senior Member, IEEE, and Paolo Dario, Fellow, IEEE,
A Survey of Glove-Based Systems and Their Applications.
• Kay M. Stanney HANDBOOK OF VIRTUAL ENVIRONMENTS Design, Implementation,
and Applications, Gesture Recognition Chapter #10 by Matthew Turk
• Daniel Thalman, Gesture Recognition Motion Capture, Motion Retargeting, and Action
Recognition
• Hatice Gunes, Massimo Piccardi, Tony Ja, 2007, Research School of Information Sciences
and Engineering Australian National University Face and Body Gesture Recognition for a
Vision-Based Multimodal Analyzer
• J. Heinzm ann and A. Zelinsky Robust Real - Time Face Tracking and Gesture Recognition
• Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva, Mosc

15