Implementation of Virtual 
Mouse and Keyboard 
B.TECH MAJOR PROJECT PROPOSAL 

SUBMITTED TO: 
A.Prof. Rajiv Kumar 

 
SUBMITTED BY: 
Vatsala Sharma (10513) 
Jasleen Kaur (10509) 
Suresh Pratap Singh (10557) 
Chhavi Kaushik (09518) 
Objective 
Developing fully functional Virtual Keyboard and Mouse, using the principles of
Image Processing. The Virtual Mouse and Keyboard are basically implementing
these devices without using any actual hardware that is traditionally used for them.

Motivation 
Technology is rapidly changing in the world today. A variety of computing devices
have become common place, and the traditional Personal Computers are no
longer the only devices where people use a keyboard and mouse.

These new devices which have gained popularity are smart phones, tablets,
notebooks, and also the laptop computers, share a very principal common feature.
That is ‘mobility’, meaning all these devices are not tied down with wires. These
devices are generally intended to be used on the go. Hence, these devices have
many issues that the desktop computer does not have. The biggest one of these
is portability. For a mobile device, size matters a lot. The device should be light-
weight, easy to handle and reasonably sized.

As a result, the traditional keyboard and mouse go out of the question. Instead,
most of these devices have moved to full touch screens and on-screen keyboards,
that are often not as effective as traditional keyboards because of size limits. Only
the laptop still has a traditional keyboard, and the mouse here too has been
replaced with a touch pad.

A better alternative to this can be the Virtual technology, which avoids the screen
space wastage caused by on-screen keyboards, provides an easier approach for
combining the best of available technologies.

PAGE 1 
Functionality 
 
Keyboard‐ 
 A full QWERTY keyboard that is able to type all small and uppercase
characters, the ten digits and some special characters.
 The keyboard has to be accurate and responsive. It should be able to
handle typing with multiple fingers.
 Multi-touch responsiveness has to be there, such as holding down the 'Shift'
or 'Function' key.

Mouse‐ 
 All basic functions have to be included - moving the cursor, scrolling, left
and right click.
 Also an easy way to shift between the keyboard and mouse mode has to
be there.

Methodology 
The basic approach is to use our fingers to do the actions of typing and using the
mouse as is done on the touch pad of a laptop, on a table top. A webcam will be
used to capture these hand motions, and this live feed will be processed to
determine the actions being done. Accordingly, proper mouse or keyboard inputs
are given to the computer.

Design Issues – 
 Most importantly, our processing program has to be able to identify the
fingers of the user’s hand and track the motion of the fingertips across the
table top.
 In case of the virtual mouse, the main issue is that there are many different
activities that are done by the mouse, such as moving the cursor, scrolling,

PAGE 2 
 dragging and left-and-right clicks. In order to differentiate between all these,
a specialized gesture has to be devised for each.
 Another issue is making the sensitivity of the mouse acceptable.
 Also, there is the question that in what area on the table top the user can
move his finger. Obviously the user’s finger should stay in view of the
camera at all times. We can demarcate a smaller area within this for
convenience.
 For the keyboard, the processing program should be able to determine
which key the user’s fingertip is currently pressing.
 The program also has to determine weather, a single key is being pressed
or multiple keys.
 For typing with multiple fingers, the program has to simultaneously track the
motions of multiple fingertips.

Development Platform – 
Visual Studio 2012, which is an integrated development environment from
Microsoft will be the main development platform. The program code will be in
Visual C++.
For the Image Processing procedures, an open source function library called
OpenCV (v. 2.4.9) will be used in conjunction with Visual Studio.

Compatibility  
 Primary goal would be to develop the program for a Windows platform, to
be used on a laptop or a desktop computer.
 According to the success, possibilities of expanding the project to other
platforms such as Android (for cell phones or tablets) can also be explored.

PAGE 3 
Future Prospects of Project 
This ‘Virtual’ technology can form the basis of many future devices. Taking the
example of a laptop computer. If the keyboard and the touchpad are replaced by
their virtual counterparts, then there is no longer a need of a two-part folding
device. The laptop would just appear as a slightly larger version of a tablet. This
confluence of technologies is what the future holds.

The hardware required for the implementation of this Virtual technology will include
a small projecting device that can project the image of a keyboard onto a flat
surface. Also, a dedicated camera unit has to be included. Both of these devices
can be mounted onto the end of a small retractable cable.

This virtual technology can also be expanded to many other devices, such as
virtual musical instruments.
 

PAGE 4