K S RANGASAMY COLLEGE OF

TECHNOLOGY

VIRUTAL MOUSE USING HAND

GESTURES

A PROJECT REPORT

Submitted by

ROSSHAN BANU S
RUBY ANGEL R
SOMAISWARIY S

in partial fulfillment of the requirement

for the award of the degree

of

B.TECH
in

INFORMATION TECHNOLOGY

K.S. RANGASAMY COLLEGE OF TECHNOLOGY

(An Autonomous Institution, affiliated to Anna University Chennai and Approved by AICTE, New Delhi)

TIRUCHENGODE – 637 215

MAY-2023
 2

K.S. RANGASAMY COLLEGE OF TECHNOLOGY

TIRUCHENGODE - 637 215

BONAFIDE CERTIFICATE

Certified that this project report titled “ VIRTUAL MOUSE USING HAND
GESTURES” is the bonafide work of ROSSHAN BANU S(1921178), RUBY
ANGEL R (1921179) and SOMAISWARIY S (1921193) who carried out the
project under my supervision. Certified further, that to the best of my knowledge the
work reported herein does not form part of any other project report or dissertation
on the basis of which a degree or award was conferred on an earlier occasion on this
or any other candidate.

SIGNATURE
HEAD OF THE DEPARTMENT
PROFESSOR
Department of Information Technology
K.S. Rangasamy College of Technology
Tiruchengode - 637 215
SIGNATURE
Dr. M.Tamilarasi, M.E., Ph.D.,
SUPERVISOR
Associate Professor
Department of Information Technology
K.S. Rangasamy College of Technology
Tiruchengode - 637 215

Submitted to

Internal Examiner External Examiner

 3

DECLARATION

We jointly declare that the project report on “VIRTUAL MOUSE USING

HAND GESTURES ” is the result of original work done by us and best of our
knowledge, similar work has not been submitted to “ANNA UNIVERSITY
CHENNAI” for the requirement of Degree of B.Tech. This project report is
submitted on the partial fulfilment of the requirement of the award of Degree of
B.Tech.

Signature

____________________
ROSSHAN BANU S

____________________
RUBY ANGEL R

____________________
SOMAISWAIRY S

Place: Tiruchengode

Date: 17-10-2022
 4

ACKNOWLEDGEMENT

We wish to express our sincere gratitude to our honourable vice-chairman

Mr.R. SRINIVASAN for providing immense facilities at our institution.

We would like to express special thanks of gratitude to our Chief Executive

Officer Dr.K. THYAGARAJAH, M.E., Ph.D., who has been the key spring of
motivation to us throughout the completion of our course and project work.

We are very proudly rendering our thanks to our Principal

Dr.R. GOPALAKRISHNAN, M.E., Ph.D., for the facilities and the
encouragement given by him to the progress and completion of our project.

We regard our sincere and heartfelt thanks to our Director of the Department
who gave us the golden opportunity to do this wonderful project.

We proudly render our immense gratitude to the Head of the Department

Dr.R. POONKUZHALI, M.E., Ph.D., for his effective leadership,
encouragement and guidance in the project.

We are highly indebted to provide our heart full thanks to our supervisor
Dr.S. SARUMATHI, B.E,M.E., Ph.D., for his valuable ideas, encouragement and
supportive guidance throughout the project.

We wish to extend our sincere thanks to all faculty members of our

Information Department for their valuable suggestions, kind co-operation and
constant encouragement for successful completion of this project.

We wish to acknowledge the help received from various Departments and

various individuals during the preparation and editing stages of the manuscript.
 5

TABLE OF CONTENTS

CHAPTER TITLE PAGE

NO.

ABSTRACT
LIST OF TABLES
LIST OF FIGURES
LIST OF SYMBOLS AND ABBREVIATIONS

1 INTRODUCTION
2 LITERATURE REVIEW

2.1.1  A survey paper on hand gesture recognition

2.1.2 hand gesture recognition based on shape parameters

2.1.3  A review of the hand gesture system: Current progress

and future directions
2.1.4  A review on hand gesture system

2.1.5 A review on hand gesture  recognition system

2.1.6 A survey paper on hand gesture recognition

2.2 Conclusions from the Literature Review

4 EXPERIMENTAL SETUP

4.1 SYSTEM ARCHITECTURE

5 RESULT AND CONCLUSION

5.1 RESULT
6 CONCLUSION
LIST OF PUBLICATIONS
APPENDIX 1
REFERNCES

ABSTRACT
 6

Gesture-controlled laptops and computers have recently gained a lot of traction.

Leap motion is the name for this technique. Waving our hand in front of our

computer/laptop allows us to manage certain of its functionalities. Over slides and

overheads, computer-based presentations have significant advantages. Audio, video,

and even interactive programmes can be used to improve presentations.

Unfortunately, employing these techniques is more complicated than using slides or

overheads. The speaker must operate various devices with unfamiliar controls (e.g.,

keyboard, mouse, VCR remote control). In the dark, these devices are difficult to

see, and manipulating them causes the presentation to be disrupted. Hand gestures

are the most natural and effortless manner of communicating. The camera’s output

will be displayed on the monitor. The concept is to use a simple camera instead of a

classic or standard mouse to control mouse cursor functions. The Virtual Mouse

provides an infrastructure between the user and the system using only a camera. It

allows users to interface with machines without the use of mechanical or physical

devices, and even control mouse functionalities. This study presents a method for

controlling the cursor’s position without the need of any electronic equipment.

While actions such as clicking and dragging things will be carried out using various

hand gestures. As an input device, the suggested system will just require a webcam.

The suggested system will require the use of OpenCV and Python as well as other

tools. The camera’s output will be presented on the system’s screen so that the user

can further calibrate it.

 1

CHAPTER 1

INTRODUCTION

The project “Mouse control using Hand Gestures” is developed aiming to better the
process of human-computer interaction. It aims to provide the user a better
understanding of the system and to let them use alternate ways of interacting with
the computer for a task. The task here is to control the mouse even from a distance
just by using hand gestures. It uses a program in python and various libraries such
as PyAutoGUI, Numpy and image processing module OpenCV to read a video feed
which identifies the users’ fingers represented by three different colors and track
their movements. It retrieves necessary data and implements it to the mouse
interface of the computer according to predefined notions. The project can be useful
for various professional and non-professional presentations. It can also be used at
home by users for recreational purposes like while watching movies or playing
games.
 2

CHAPTER 2

2.1 LITERATURE REVIEW

2.1.1“BHUMIKA NANDWANA; SATYANARAYAN TAZI;

SHEIFALEE TRIVEDI; DINESH KUMAR; SANTOSH KUMAR” A
survey paper on hand gesture recognition

Hand gesture recognition system has a good attention now days because of
easy interaction between human and machine. The focus of developing hand
gesture is to create a better communication between human and computer for
conveying information. This paper presents a survey on recent technology of
hand gesture recognition both static and dynamic. It shows all method that
was used for hand gesture recognition in different research paper.

2.1.2 “MEENAKSHI PANWAR” hand gesture recognition based on shape

parameters

Pattern recognition and Gesture recognition are the growing fields of research.
Being a significant part in non-verbal communication hand gestures are playing
vital role in our daily life. Hand Gesture recognition system provides us an
innovative, natural, user friendly way of interaction with the computer which is
more familiar to the human beings. Gesture Recognition has a wide area of
application including human machine interaction, sign language, immersive
game technology etc. By keeping in mind the similarities of human hand shape
 3

with four fingers and one thumb, this paper aims to present a real time system for
hand gesture recognition on the basis of detection of some meaningful shape
based features like orientation, centre of mass (centroid), status of fingers, thumb
in terms of raised or folded fingers of hand and their respective location in
image. The approach introduced in this paper is totally depending on the shape
parameters of the hand gesture. It does not consider any other mean of hand
gesture recognition like skin color, texture because these image based features
are extremely variant to different light conditions and other influences. To
implement this approach we have utilized a simple web cam which is working
on 20 fps with 7 mega pixel intensity. On having the input sequence of images
through web cam it uses some pre-processing steps for removal of background
noise and employs K-means clustering for segmenting the hand object from rest
of the background, so that only segmented significant cluster or hand object is to
be processed in order to calculate shape based features. This simple shape based
approach to hand gesture recognition can identify around 45 different gestures
on the bases of 5 bit binary string resulted as the output of this algorithm. This
proposed implemented algorithm has been tested over 450 images and it gives
approximate recognition rate of 94%.

“
2.1.3 NORAINI MOHAMED; MUMTAZ BEGUM MUSTAFA;
NAZEAN JOMHARI” A review of the hand gesture system: Current
progress and future directions

This paper reviewed the sign language research in the vision-based hand
gesture recognition system from 2014 to 2020. Its objective is to identify the
progress and what needs more attention. We have extracted a total of 98
articles from well-known online databases using selected keywords. The
review shows that the vision-based hand gesture recognition research is an
active field of research, with many studies conducted, resulting in dozens of
articles published annually in journals and conference proceedings. Most of
 4

the articles focus on three critical aspects of the vision-based hand gesture
recognition system, namely: data acquisition, data environment, and hand
gesture representation. We have also reviewed the performance of the vision-
based hand gesture recognition system in terms of recognition accuracy. For
the signer dependent, the recognition accuracy ranges from 69% to 98%, with
an average of 88.8% among the selected studies. On the other hand, the signer
independent’s recognition accuracy reported in the selected studies ranges
from 48% to 97%, with an average recognition accuracy of 78.2%. The lack in
the progress of continuous gesture recognition could indicate that more work is
needed towards a practical vision-based gesture recognition system.

2.1.4“JAYESH S. SONKUSARE; NILKANTH B. CHOPADE;

RAVINDRA SOR; SUNIL L. TADE” A review on hand gesture system

From the ancient age, gesture was the first mode of communication, after the
evolution of human civilization they developed the verbal communication,
but still non-verbal communication is equally significant. Such non-verbal
communication is not only used for physically challenged person but also it
can be efficiently used for various applications such as 3D gaming, aviation,
surveying, etc. This is the best method to interact with computer without any
peripheral devices. Many Researchers are still developing robust and efficient
new hand gesture recognition techniques. The major steps associated while
designing the system are: data acquisition, segmentation and tracking, feature
extraction and gesture recognition. There're different methodologies associated
with several substepspresent at each step. A various segmentation and Tracking,
feature extraction and recognition techniques are studied and analyzed. This
paper reviews the comparative study of various hand gesture recognition
techniques which are presented up-till now.
 5

2.1.5 “ROSHNEE MALTANI” A review on hand gesture recognition

system

Motion controlled PCs and PCs have as of late built up momentum. Jump
movement is the name for this method. Waving our hand before our PC/PC
permits us to deal with sure of its functionalities. Over slides and overheads,
PC based introductions enjoy critical benefits. Sound, video, and,
surprisingly, intuitive projects can be utilized to further develop introductions.
Sadly, utilizing these strategies is more convoluted than utilizing slides or
overheads. The speaker should work different gadgets with new controls (e.g.,
console, mouse, VCR controller). In obscurity, these gadgets are hard to see,
and controlling them makes the show be upset. Hand signals are the most
normal and easy way of imparting. The camera's result will be shown on the
screen. The idea is to utilize a basic camera rather than a work of art or
standard mouse to control mouse cursor capabilities. The Virtual Mouse gives
a foundation between the client and the framework utilizing just a camera. It
permits clients to communicate with machines without the utilization of
mechanical or actual gadgets, and even control mouse functionalities. This
study presents a strategy for controlling the cursor's situation without the need
of any electronic gear. While activities, for example, clicking and hauling
things will be completed utilizing different hand motions. As an information
gadget, the recommended framework will simply require a webcam. The
recommended framework will require the utilization of OpenCV and Python
as well as different instruments. The camera's result will be introduced on the
framework's screen with the goal that the client can additionally adjust it.
 6

2.1.6 “RAMAZAN OZGUR DOGAN” A survey paper on hand gesture

recognition

Virtual mouse carried out with hand motion following in view of picture is
one of concentrates in human-PC connection. In this study that human-PC
communication is executed with virtual mouse is purposed. This review
comprises of three principal steps that are hand motion following, highlights
of hand district extraction and grouping of these elements. In this review hand
signal following is created with Camshift (Consistently Versatile Mean Shift)
calculation, elements of hand motions are extricated with sack of visual words
and these highlights are arranged with help vector machines. Nitty gritty tests
are performed to think about outcome of following, highlights extraction and
classisfication techniques and that this framework works effectively is shown.

2.2 CONCLUSIONS FROM THE LITERATURE REVIEW

The existing system proposed an awfully high demanding cost and resources.
Because of this, there have not been many users who could afford it.There are many
software or application are present on the internet which claims that they can
control the mouse without touching it but on the ground check reality they do not
the same as they say instead of they use controlling ofmouse virtually by using
coloured tape on the finger or using colour monitoring techniques that inappropriate
for anyone to manage those kinds of stuff. And at the end, none can get the final
accurate outcome or fail to provide the smoothing in the movement of virtual mouse
that creates hindrance in using it and disturb the movement of that mouse.
 7

PROBLEM STATEMENT

System since at the beginning there will be a little difficulty

• The user has to give some time to get adopted this type of controlling the
cursor. Hence, it takes some time to get familiar with the system
• The module can track only one hand so there might be a problem when there is
more than one hand in the frame
• The hand tracking software requires optimum level of light and brightness
which limits the use of the software in places with low light

PROPOSED SYSTEM

I am using my hand as a virtual mouse than can do everything that a mouse does
• without even touching your system
• I am using the webcam of my system to detect my hands
• It will then create a bounding box around my hand and focus on two fingers:
• The forefinger and the middle finger.
• The fore finger will act as a cursor and moving it around, we will be moving the
cursor around.
• Now, in order to successfully click using hand tracking, it is detecting the distance
between the fore finger and the
middle finger
• If they are joined together, then it will perform a click
 8

CHAPTER 3

SYSTEM ANALYSIS AND DESIGN

3.1 SOFTWARE TOOLS

Introduction

Python is a high-level programming language designed to be easy to read and

simple to implement. It is open source, which means it is free to use, even for
commercial applications. Python can run on Mac, Windows, and Unix systems and
has also been ported to Java and .NET virtual machines. Python is a fairly old
language created by Guido Van Rossum. The design began in the late 1980s and
was first released in February 1991. Python is considered a scripting language, like
Ruby or Perl and is often used for creating Web applications and dynamic Web
content. It is also supported by a number of 2D and 3D imaging programs, enabling
users to create custom plug-ins and extensions with Python. Examples of
applications that support a Python API include GIMP, Inkscape, Blender, and
Autodesk Maya. Scripts written in Python (.PY files) can be parsed and run
immediately. They can also be saved as a compiled programs (.PYC files), which
are often used as programming modules that can be referenced by other Python
programs.In late 1980s, Guido Van Rossum was working on the Amoeba
distributed operating system group. He wanted to use an interpreted language like
ABC (ABC has simple easy-to-understand syntax) that could access the Amoeba
system calls. So, he decided to create a language that was extensible. This led to a
design of new language which was later named Python. No. It wasn't named
after a dangerous snake. Rossum was fan of a comedy series from late seventies.
The name "Python" was adopted from the same series "Monty
Python's Flying Circus".
 9

Features of python:

1. A simple language which is easier to learn

Python has a very simple and elegant syntax. It's much easier to read and
write Python

programs compared to other languages like: C++, Java, C#. Python makes
programming fun

and allows you to focus on the solution rather than syntax.

If you are a newbie, it's a great choice to start your journey with Python.

2. Free and open-source

You can freely use and distribute Python, even for commercial use. Not only can
you use and

distribute softwares written in it, you can even make changes to the Python's
source code.

Python has a large community constantly improving it in each iteration.

3. Portability

You can move Python programs from one platform to another, and run it without
any

changes.

It runs seamlessly on almost all platforms including Windows, Mac OS X and

Linux.

4. Extensible and Embeddable

Suppose an application requires high performance. You can easily combine pieces
of C/C++

or other languages with Python code.

 10

This will give your application high performance as well as scripting capabilities
which other

languages may not provide out of the box.

5. A high-level, interpreted language

Unlike C/C++, you don't have to worry about daunting tasks like memory
management,

garbage collection and so on.

Likewise, when you run Python code, it automatically converts your code to the
language

your computer understands. You don't need to worry about any lower-level
operations.

6. Large standard libraries to solve common tasks

Python has a number of standard libraries which makes life of a programmer much
easier

since you don't have to write all the code yourself. For example: Need to
connect MySQL

database on a Web server? You can use MySQLdb library using import MySQLdb .

Standard libraries in Python are well tested and used by hundreds of people. So you
can be

sure that it won't break your application.

7. Object-oriented

Everything in Python is an object. Object oriented programming (OOP) helps you

solve a complex problem intuitively. With OOP, you are able to divide these
complex problems into smaller sets by creating objects.
 11

Python Applications:

You can create scalable Web Apps using frameworks and CMS (Content
Management System) that

are built on Python. Some of the popular platforms for creating Web Apps are:
Django, Flask,

Pyramid, Plone, Django CMS.

Sites like Mozilla, Reddit, Instagram and PBS are written in Python.

Scientific and Numeric Computing

There are numerous libraries available in Python for scientific and numeric
computing. There are

libraries like: SciPy and NumPy that are used in general purpose computing. And,
there are specific

libraries like: EarthPy for earth science, AstroPy for Astronomy and so on.

Also, the language is heavily used in machine learning, data mining and deep
learning. Creating software Prototypes Python is slow compared to compiled
languages like C++ and Java. It might not a good choice if resources are limited and
efficiency is a must. However, Python is a great language for creating prototypes.
For example: You can use Pygame (library for creating games) to create your
game's prototype first. If you like the prototype, you can use language like C+
+ to create the actual game. Good Language to Teach Programming Python is used
by many companies to teach programming to kids and newbies. It is a good
language with a lot of features and capabilities. Yet, it's one of the easiest
language to learn because of its simple easy-to-use syntax. Python is a terrific
language. The syntax is simple and code length is short which makes is easy to
understand and write.If you are getting started in programming, Python is an
awesome choice. You will be amazed how much you can do in Python once you
know the basics.It's easy to overlook the fact that Python is a powerful
language. Not only is it good for learning programming, it's also a good
 12

language to have in your arsenal. Change your idea into a prototype or create games
or get started with data Science, Python can help you in everything to get started.

Advantages:

1. Time-wasting matters of style, especially regarding enclosing curly-bracket

positioning

blocks, simply go away.

2. Whenever you’re faced with a problem and are figuring out how to do it, there
will be

multiple well-documented ways.

3. You can become productive in Python fairly quickly even as a beginner, yet it
will serve you

in industry like a champ too!

Now what do I mean by this? The clearest example is the non-use of curly braces—
or brackets of any

sort, for that matter—as code block delimiters. One of the most debated matters of
programming style

in BCPL-derivative languages such as C and Java is “indent styles”, or how to

arrange those curly-

brackets for maximum readability—which happens to make absolutely no

difference in how your

program actually runs. Curly braces were a BCPL invention (a precursor to C) to

help

compilers—not developers. Programmer time lost on these matters of style can now
be measured in
 13

centuries. That curly braces have propagated to like every other “serious” language
is one of the

biggest and saddest facts in the world of information technology.

Ruby, Visual Basic and most non-C derivative languages instead of curly brackets
use “keywords”

such as Ruby’s “def” and “end” for code blocks, instead of using curly-brackets.
Python takes it one

step further and only uses a keyword only at the beginning. The rest of the code
block is contained

merely by virtue of line indenting. That white space matters is perhaps the most
controversial

(contrary to most other languages) and pragmatic thing about Python—and drives
plenty of people

crazy who want to dis’ Python on that basis—a position that fills with glee those of
us who’ve traded

in the pain of curly brackets for the clarity of whitespace long ago. However, the
price of Python’s

innovation is the required use of a colon at the end of a function or class—the

leaving-off-of-which is

probably among the largest of Python newbie mistakes.

Benefits of Python

Python is an dynamic object-oriented programming language that can be compared

with Java and
 14

Microsoft's .NET-based languages as a general-purpose substrate for many

kinds of software

development. It offers strong support for integrating with other technologies, higher
programmer

productivity throughout the development life cycle, and is particularly well suited
for large or

complex projects with changing requirements.

Python is the most rapidly growing open source programming language. According
to InfoWorld its

user base nearly doubled in 2004, and currently includes about 14% of all
programmers.

Python is being used in mission critical applications in the world's largest

stock exchange, forms the

basis for high end newspaper websites, runs on millions of cell phones, and is used
in industries as

diverse as ship building, feature length movie animation, and air traffic control.

Python is available for most operating systems, including Windows, UNIX, Linux,
and Mac OS

Key Strengths

Python's clean object-oriented design and extensive support libraries offer two
to ten fold the

programmer productivity seen with languages like C, C++, C#, Java, VB, and Perl.
 15

Integration Of all types, including Enterprise Application Integration (EAI). Python

makes it easy to

develop Web services, can invoke COM or CORBA components, calls directly to
and from C, C++,

or Java code (via Jython), provides powerful process control capabilities,

implements all common

internet protocols and data formats, processes XML and other markup languages,
can be embedded as

a scripting language, and runs from the same byte code on all modern operating
systems.

Network intensive applications And complex multi-protocol network applications

can be built on

Twisted, a development framework well suited to running large numbers of

concurrent network,

database, and inter-process communication links within the same process.

Web development from simple CGI scripting to high-end web application

development with mega-

frameworks such as Django and Turbogears, the Zope application server, Plone
content management

system, Quixote web application framework, or a even a home-grown solution

based on Python's

extensive and easy to use standard libraries. Python provides interfaces to most
databases, powerful
 16

text processing and document processing facilities, and plays well with other web
technologies.

Numeric and scientific applications make use of the Python Imaging Library, VTK
and MayaVi 3D

Visualization Toolkits, Numeric Python, Scientific Python and many other tools
available for

numeric and scientific applications. Many of these are supported by the Enthought
Python

Distribution.

Application scripting is a snap with Python's tight integration with C/C++ and
Java. Python was

designed from the ground up to be embeddable and serves as an excellent choice as

a scripting

language for customization or extension of larger applications.

Software Testing benefits from Python's strong integration and text processing
capabilities, and

Python comes with its own unit testing framework.

Desktop development using wxPython, PyQt, or PyGtk for high-quality GUI

applications. Protect

your investment by basing on open technologies, with deployment to most

operating systems.

Support for other GUI frameworks, such as MFC, Carbon, Delphi, X11, Motif, and
Tk, is also

available.

Prototyping
 17

Python is quick and easy, and often results in development of the final system in
Python. The agile

nature of the language, and the ease of refactoring code makes for rapid
development directly from

the initial prototype.

The Open Source Advantage

Because it has been developed as open source by thousands of contributors from

around the world,

Python is very well designed, fast, robust, portable, and scalable. With an
uncluttered, easy-to-learn

syntax and well-developed advanced language features, Python often exceeds the
capabilities of

comparable commercially available solutions.

The open source license for Python allows unrestricted use, modification, and
redistribution of the

language or anything that is based on it, commercially or otherwise. Full source is

available and there

are no license costs. Support is available for free, from a rich set of internet-based
resources, and

from organizations in the business of providing paid support to Python users.

 18

FLOW DIAGRAM
 19

TECHNIQUES
The flow model shows the working of the system with different functions. The
system will first take the input of image from the system camera; it’ll also convert
the video captured from the web camera into frames. It'll then resize the input image
so that the segmentation can happen to detect the points on the image. It'll de noise
from the image and begin showing the middle radius of the image of the specified
finger tips. The radius points are going to be centered on the image of the finger tip.
The finger tips will now start moving in keeping with the movement of the fingers.
It'll detect the points of the radius; we will now manipulate the cursor with fingertip
movement.
Autopy .Real time video captured by the Webcam is processed and only the two
finger tips are extracted with the use of those two fingers we can use our mouse.
Their centres are measured by using the system webcam finger tips moments, and
therefore the action to be taken is decided supported their relative positions and
their respective distance give us the idea how far we should . The first goal is to use
the function cv2.VideoCapture ().This function uses to capture the live stream video
on the camera. OpenCV will create a very easy interface to try to this. To capture a
image we want to form an video capture object. We then covert this captured
images into HSV format.

CHAPTER 4

4.1 SYSTEM ARCHITECTURE

 20

IMPLEMENTATION
We all use new technology development in our everyday life. Including our devices
similarly after we speak about technology the most effective example may be a
computer. A computer has evolved from a really low and advanced significantly
over the decades since they originated. However we also use the identical setup,
which have a mouse and keyboard.. Though the technology have made many
changes within the development of computers like laptop where the camera is now
an integrated a part of the pc. We still have a mouse which is either integrated or an
external device. This is how we've got encounter the implementation a brand new
technology for our mouse where we can control the pc by finger tips and this
method is thought as Hand Gesture Movement. With the help of our fingers, we will
be ready to guide our cursor.

For this project we've used Python because the base language as it is an open source
and simple to grasp and environment friendly. The packages that are required here
is Autopy and OpenCV. Autopy is a Python module for programmatically
controlling the mouse and keyboard. OpenCV through which we can control mouse
events. Processing to extract required data so adds it to the computer's mouse
interface in keeping with predefined notions. Python is employed to write the file. It
uses of the cross platform image processing module OpenCV and implements the
mouse actions using Python specific library Autopy .Real time video captured by
the Webcam is processed and only the two finger tips are extracted with the use of
those two fingers we can use our mouse. Their centres are measured by using the
system webcam finger tips moments, and therefore the action to be taken is decided
supported their relative positions and their respective distance give us the idea how
far we should . The first goal is to use the function cv2.VideoCapture ().This
function uses to capture the live stream video on the camera. OpenCV will create a
very easy interface to try to this. To capture a image we want to form an video
capture object. We then covert this captured images into HSV format.

RESULTS AND DISCUSSION

5.1 RESULT

I am using my hand as a virtual mouse than can do everything that a mouse

does without even touching your system. I am using the webcam of my
system to detect my hands. It will then create a bounding box around my hand
and focus on two fingers: The fore finger and the middle finger. The fore
 21

finger will act as a cursor and moving it around, we will be moving the cursor
around. Now, inorder to successfully click using hand tracking, it is detecting
the distance between the fore finger and the middle finger. If they are joined
together, then it will perform a click.

OUTPUT
 22

CONCLUSION
In the end, we conclude that we learn a lot of things during the development of the
projects like how to work with open computer vision and its library, dynamic
Applications, databases, and python language. In a nutshell, it can be summarized
that the future scope of the project circles is to provide mouse virtually those
students who are physically disabled and also that students who want to get rid of
the physical mouse.
 23

LIST OF PUBLICATIONS

APPENDIX 1

APPLICATION AIMS
a. Obtain input video feed
b. Retrieve useful data from the image to be used as input
c. Filter the image and identify different colours.
d. Track the movement of colours in the video frame.
e. Implement it to the mouse interface of the computer according to predefined
notions for
Mouse pointer control.
 24

REFERENCES

[1] Shivendra Singh, Real Time Hand Gesture Recognition Using Finger Tips,
International Research Journal of Engineering and Technology (IRJET), Volume:
05 Issue: 06 | June -2018 page 1420

[2] Simran Shah, A Vision Based Hand Gesture Recognition System using
Convolutional Neural Networks, International Research Journal of Engineering and
Technology (IRJET), Volume: 06 Issue: 04 | Apr 2019 page 2570

[3] Rutuja J, Hand Gesture Recognition System Using Convolutional Neural

Networks, InternationalResearch Journal of Engineering and Technology
(IRJET),Volume: 06 Issue: 04 | Apr 2019 page 4508

[4] Multimodal Hand Gesture dataset https://www.kaggle.com/c/multi-modal-

gesturerecognition

[5] Tasneem Gorach, DEEP CONVOLUTIONAL NEURAL NETWORKS- A

REVIEW, International Research Journal of Engineering and Technology (IRJET),
Volume: 05 Issue: 07 | July-2018 page 439

[6] OpenCV. Open Source Computer Vision Library. 2015, https://opencv.org/

[7] https://docs.opencv.org/2.4/modules/imgproc/doc/m
otion_analysis_and_object_tracking.html

[8] Rafiqul Zaman Khan, COMPARATIVE STUDY OF HAND GESTURE

RECOGNITION SYSTEM, Department of Computer Science, A.M.U., Aligarh,
India
[9] Diederik P. Kingma, Jimmy Lei Ba, ADAM: A METHOD FOR STOCHASTIC
OPTIMIZATION, Published as a conference paper at ICLR 2015
 25

[10] Jiang L., Wang D., Cai Z., Yan X., “Survey of Improving Naive Bayes for
Classification”, Advanced Data Mining and Applications Lecture Notes in Computer
Science, 4632, (2007) 134-145.

[11] Asaari M. S. M., Suandi S. A., “Hand Gesture Tracking System Using Adaptive
Kalman Filter”, Intelligent systems design and applications (ISDA), International
Conference on IEEE, Aralık 2010, Cairo, Bildiriler Kitabı, 166 -171.

[12] Isard M., Blake A., "CONDENSATION-- conditional density propagation of visual
tracking". International Journal of Computer Vision 29 (1): 5–28. [9] Zhou H., Xie L.,
Fang X., “Visual Mouse: SIFT Detection and PCA Recognition”, Computational
Intelligence and Security Workshops (CISW). International Conference on IEEE, Aralık
2007, Harbin, Bildiriler Kitabı, 263-266.

[12] Dai J., Song W., Pei L., Zhang J., “Remote Sensing Image Matching via Harris
Detector and SIFT Discriptor”, Image and Signal Processing (CISP), International
Conference on IEEE, Ekim 2010, Yantai, Bildiriler Kitabı, 2221-2224.

[13] Burges C. J. C., “A Tutorial on Support Vector Machines for Pattern Recognation”,
Data Mining and Knowledge Discovery, 2, 2 (1998) 121-167. [12] Abe S., “Support
Vector Machines for Pattern Classification”, Second Edition, Springer, Kobe, 2010.