Minor Project on

Sign Language Recognition Using Machine Learning

By

Group 9
Syead Maaz Ahmed (201900007)
Farheen (201900014)
Ebbani Thapa (201900040)

Under the guidance of

Mrs. Pratima Sarkar, Assistant Professor

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING SIKKIM MANIPAL INSTITUTE OF TECHNOLOGY

(A constituent college of Sikkim Manipal University)
MAJITAR, RANGPO, EAST SIKKIM-737136
 ABSTRACT
• Inability to speak is true disability. Speech impairment is a disability that affects an
individual’s ability to communicate using speech and hearing.
• Mode of communication such as sign language is used by people affected by this
impairment. But there exists a challenge for non-signers to communicate with
signers, although there has been several advancements in the technology.
• Aiding the cause, deep learning, and computer vision can be used too to make an
impact on this cause.
• The major focus of this work is to create a deep learning-based application that
offers sign language translation to text thereby aiding communication between
signers and non-signers.
 INTRODUCTION

• The world can hardly live without communication, no matter whether it is in the
form of text, voice or visual expression.
• The communication among the deaf and dumb people is carried by text and visual
expressions.
• Gestural communication is always in the scope of confidential and secure
communication.
• Hands and facial parts are immensely influential to express the thought of human
in confidential communication.
 LITERATURE SURVEY
SL No. Author Paper and Publication Details Findings Relevance to the
project
1 Ameen, SA and A convolutional neural network to classify • Exploration and presentation of • The utilization of
Vadera, S American Sign Language fingerspelling the applicability of deep deep learning
from depth and colour images. learnings for interpreting sign architecture to
(University of Salford Manchester (2017)) language. recognise the
• The paper develops a hand gestures of
convolutional neural network sign language
aimed at classifying
fingerspelling images using
both image intensity and depth
data.

2 L. Tolentino, R. Static Sign Language Recognition Using • The overall letter recognition • Deep
Juan, August C. , Deep Learning accuracy of the system was understanding of
Maria A. (International Journal of Machine Learning achieved by getting the average Convolutional
Pamahoy, J. and Computing, Vol. 9, No. 6, December of each letter’s accuracy. Neural Network
Forteza, and 2019) • The system attained 90.04% (CNN) as the
Xavier O. Gracia accuracy. recognizer of the
• It also attained an average time system.
of 4.31 second real-time letter
recognition of hands that were
not in the dataset.
SL No. Author Paper and Publication Details Findings Relevance to the project

3. Nikhil American Sign Language Alphabet • The researchers found the • Conceptions of how
Kasukurthi, Brij Recognition using Deep Learning maximum training accuracy of the lighting conditions
Rokad, Shiv (Vellore Institute of Technology University 87.47% with the validation and distance of the
Bidani, Dr. Aju (2019) ) accuracy to be 83.29%. image from the
Dennisan • The network learned the ASL camera should not
that allowed it to predict the affect the outcome of
sign language in real time. the prediction
thereby not affecting
the accuracy rate.

4. R.S. Sabeenian, Sign Language Recognition Using Deep • Introduction of the CNN based • Usage of Custom CNN
S. Sai Learning and Computer Vision approach for the recognition Model Architecture
Bharathwaj ( Journal of Advanced Research in and classification of the sign which consists of 11
and , M. Dynamical and Control Systems, May, language using computer different layers.
Mohamed 2020) vision, with better accuracy and
Aadhil considerably low false positives.
 PROBLEM DEFINITION

• According to 2011 census of India, there are 63 million people which sums up to
6.3% of the total population, who are suffering from hearing problems
• Sign languages is not just ‘natural language represented by sign’ or not just hand
representation of the words as it is but rather it is the representation of meaning.
• Understanding the exact context of symbolic expressions of deaf and dumb people is
the challenging job in real life until unless it is properly specified.
• Hence there is a need of a system which recognizes the different signs, gestures and
conveys the information to the normal people.
 SOLUTION STRATEGY

• The major reason to develop this system is to make communication through the
internet easier for the deaf and mute community.
• It can offer an opportunity for deaf people to talk with vocal humans without the need
of an interpreter.
• Steps:
i. Image Capture:- This is the first step in sign recognition. Web camera issued to
capture the hand gesture.
ii. Image Preprocessing:- Image preprocessing contains cropping, filtering,
brightness &contrast adjustment & many more.
iii. Feature Extraction:- Feature extraction is a very useful step to create the
database of sign recognition.
FLOW CHART
 GANTT CHART
Activity Time Frame
November December January Febuary March

Problem Identification

Feasibility Study

Literature Survey

SRS and Design

Coding

Testing

Documentation

Proposed Activity
Approved Activity
 REFERENCES
[1] Amen, S., & Vadera, S. (2017). A convolutional neural network to classify American Sign Language
fingerspelling from depth and colour images (University of Salford Manchester (2017))

[2] L. Tolentino, R. Juan, August C. , Maria A. Pamahoy, J. Forteza, and Xavier O. Gracia. Static Sign
Language Recognition Using Deep Learning (International Journal of Machine Learning and
Computing, Vol. 9, No. 6, December 2019)

[3] Nikhil Kasukurthi, Brij Rokad, Shiv Bidani, Dr. Aju Dennisan. Sign Language Recognition Using
Deep Learning and Computer Vision (Vellore Institute of Technology University (2019) )

[4] R.S. Sabeenian, S. Sai Bharathwaj and , M. Mohamed Aadhil. Sign Language Recognition Using
Deep Learning and Computer Vision ( Journal of Advanced Research in Dynamical and Control
Systems, May, 2020)

[5] Prof. Radha S. Shirbhate, Mr. Vedant D. Shinde, Ms. Sanam A. Metkari, Ms. Pooja U. Borkar, Ms.
Mayuri A. Khandge Using Sign language Recognition Using Machine Learning Algorithm (International
Research Journal of Engineering and Technology (IRJET)) (2020)