Marineni Akhila

AI And ML Final Project

04/15/2023 Annual Review 1

SIGN LANGUAGE RECOGNITION

04/15/2023 Annual Review 2

AGENDA
 Sign Language is a form of communication used primarily by people hard of
hearing or deaf. This type of gesture-based language allows people to convey
ideas and thoughts easily overcoming the barriers caused by difficulties from
hearing issues.

 Computer recognition of sign language deals from sign gesture acquisition and
continues till text/speech generation.

 The Main Agenda of Sign Language recognition system is to provide an

efficient and accurate way to convert sign language into text or voice.

04/15/2023 Annual Review 3

 PROBLEM STATEMENT
The Objective of this Project is to Identify the Symbolic Expression
through images so that the communication gap between a normal and
Hearing impaired person can be easily bridged.

The Aim is to develop a user friendly Human Computer Interfaces

(HCI) where the computer understands the human sign language.

The deaf or dumb person should provide a gesture or sign image

to the system. The system evaluates the sign input with matlab image
processing technique and classifies the input to the recognized
identification in the form of text or voice.

04/15/2023 Annual Review 4

 PROJECT OVERVIEW
 In our project we basically focus on producing a model which can recognize Fingerspelling
based hand gestures in order to form a complete word by combining each gesture with the
help of:

 TensorFlow : TensorFlow is an open source software library for numerical computation. It

can be used across a range of tasks but has a particular focus on training and inference of
deep neural networks. TensorFlow is widely used in Machine Learning.

 Keras : It is a high-level neural networks library written in python that works as a wrapper
to TensorFlow. It is used in cases where we want to quickly build and test the neural
network with minimal lines of code.

 OpenCV(Open Source Computer Vision):It is an open source library is mainly used for
image processing, video capture and analysis for features like face and object recognition.
04/15/2023 Annual Review 5
 WHO ARE THE END USERS?
 Most Commonly used by deaf and dumb people who have hearing or speech problems to
communicate among themselves or with normal people.

 Communication is the process of exchange of thoughts and messages in various ways such
as speech, signals, behavior and visuals.

 Deaf and dumb people make use of their hands to express different gestures to express
their ideas with other people.

 Gestures are the nonverbally exchanged messages and these gestures are understood with
vision. This nonverbal communication of deaf and dumb people is called sign language.

04/15/2023 Annual Review 6

YOUR SOLUTION AND ITS VALUE PROPOSITION

The Solution is to Provide a model or Project that can detect Sign language
gestures by using CNN(Conventional Neural Networks)which is modest way to solve
the problem; it comprises of input ,hidden and output layers with nuances in each
layer .Sign language is simplest way to understand the language of impaired one’s.

This solutions may give assertive and strong suit to the impaired people in which
they can share their feelings and words easily to others where input is given as
person gestures which is processed and converted to text by image processing.

04/15/2023 Annual Review 7

THE WOW IN YOUR SOLUTION
 This Model can be enhanced more by incorporating various nuances such as
more diverse gestures from different users in different backgrounds.

 This May also lead to two-way communication model which can be used for
direct communication .

 This solution can develop model which convert the gestures into different
languages which can understand by different users.

04/15/2023 Annual Review 8

MODELLING

04/15/2023 Annual Review 9

 RESULTS
Hand signs in a live video stream are converted to text. The proposed approach uses a web camera to
capture a picture/set of frames of a gesture and apply grayscale conversion to them. This enables the
binary decision of finding the interested portion of the image.
After applying a filter on the grayscale image as a smoothing operation, thresholding is performed using
a binarization method to separate the hand sign from the background. Based on the width and height of
the hand, the horizontal and vertical orientation of the hand is determined.
Following this, the bounding box and finger peaks in the image are determined. Based on the finger
peak positions and orientation of the hand, a 5-bit representation of the hand sign is concluded. This 5
bit value, enables a comparison of signs with the corresponding letter/number notations. The sign
detected is then converted to text that can be understood by the common public.

Video of project demonstration:

https://drive.google.com/file/d/1_ElFBVzxyVecQVM-gETZYwi_V9vj14bz/view?usp=sharing
https://drive.google.com/file/d/1_ElFBVzxyVecQVM-gETZYwi_V9vj14bz/view?usp=sharing
04/15/2023 Annual Review 10