Virtual Mouse : Software Engineering Project

Furqan Ali Khan 148

Intikhab Hussain154
Ammar Saleem144

Table of Index
Introduction .................................................................................................................................................. 3
Virtual Mouse:........................................................................................................................................... 3
Why VIRTUAL MOUSE is needed .................................................................................................................. 3
Existing System ............................................................................................................................................. 4
TrackBall: ................................................................................................................................................... 4
Disadvantages ....................................................................................................................................... 4
Mechanical Mouse: ................................................................................................................................... 4
Disadvantages ....................................................................................................................................... 4
Existing System Contd. ............................................................................................................................ 5
Optical Mouse : ......................................................................................................................................... 5
Disadvantages ....................................................................................................................................... 5
Physical Mouse Constraints ...................................................................................................................... 5
Proposed System .......................................................................................................................................... 6
Features of VIRTUAL MOUSE: ................................................................................................................... 6
Flow Chart ..................................................................................................................................................... 7
Final Flow Chart ........................................................................................................................................ 7
Hardware and Software Requirements ........................................................................................................ 8
Hardware Requirements-.......................................................................................................................... 8
Software Requirements- ........................................................................................................................... 8
References .................................................................................................................................................. 10

Introduction
There is no other more popular navigation/interaction device
than the mouse.
Overcoming the restrictions a corporeal device has would
enrich the experience of interaction.
Keeping this in mind, we present an idea of implementing a
virtual mouse system as an alternative.

Virtual Mouse:
User makes a specified hand gesture that is captured by a
camera.
Object recognition techniques are used to extract information
from the capture.
This is then translated to some meaningful event on the screen.

Why VIRTUAL MOUSE is needed

Mouse is a physical device subject to mechanical wear and tear.
It is not easy to adapt to different environments and is often
limited by environment.
Limited functions even in present operational environments.
Virtual Mouse hopes to fulfill these requirements of the user
by overcoming normal physical barriers.

Existing System
TrackBall:
The user rolls the ball with the thumb, fingers, or the palm of
the hand to move a cursor.
Large tracker balls are common on CAD workstations for easy
precision.
Before the advent of the touchpad, small trackballs were
common on portable computers.
Disadvantages

Usually not as accurate as a mouse.

Ball mechanism of trackballs requires more frequent cleaning
than a mouse.
Not very user friendly.

Mechanical Mouse:
A single ball that could rotate in any direction.
As part of the hardware package of the Xerox Alto computer.
Detection of the motion of the ball was light based with the help
of chopper wheels.
Disadvantages

Cannot provide high precision performance.

Have specific surface requirements to operate.
Needs more desk space when compared with a trackball.
4

Existing System Contd.

Optical Mouse :
Uses a light-emitting diode and photodiodes to detect movement
relative to the underlying surface.
Digital image correlation, a technology pioneered by the defense
industry for tracking military targets.
Use image sensors to image naturally occurring texture in
materials such as wood, cloth, mouse pads and
Image captures in continuous succession and comparison to
determine mouse movement.
Disadvantages

Special hardware required.

Again, specific surface requirements.

Physical Mouse Constraints

Your mouse has moved. Windows must be restarted for the
change to take effect.

Proposed System
Features of VIRTUAL MOUSE:
Any new product should either make human life more comfortable, more
productive or more fun.
Provides greater flexibility than the existing system.
Can provide more functions depending on the choice of object.
Easy to modify and adapt
Less prone to physical damage due to absence of a fixed physical device.
Avoid the mouse-related wrist damage like CTS & RSI.
Also, there is a certain degree of fun & entertainment associated with the
whole idea.

It is divided into the following modules:

Module 1 (Image Acquisition).
Module 2 (Object Recognition).
Module 3 (Object Tracing and Information Retrieval).
Module 4 (Point Coordinate Calculation and Motion Analysis).
Module 5 (Setting Cursor Position)
Module 6 (Event Generation)

Flow Chart
USER

VIRTUAL
MOUSE

OS

Final Flow Chart

IMAGE
ACQUISITION

EVENT
GENERATION

OBJECT
RECOGNITION

SETTING CURSOR
POSITION

TRACE OBJECT

COORDINATE
CALCULATIONS

Hardware and Software Requirements

Hardware Requirements Intel Pentium D processor 1.8 GHz or AMD Athlon X2 processor
1.8 GHz or higher
3 GB RAM
5 GB HDD space
Peripheral webcam at least 30 frames/second, 640x480 resolution
Software Requirements Windows XP x86 or higher (for x86 environment)
Windows XP professional x64 or higher (for x64 environment)
.NET framework 3.5 or higher
Visual Studio 2008 professional
EmguCV library (wrapper of OpenCV library for .NET framework)
EmguCV library 64 bit binaries (for developing on x64
environment)
Webcam drivers (device specific)
MATLAB

THANK YOU

References

1. Richard E. Woods, Rafael C. Gonzalez, Digital Image Processing, Pearson Education Asia, 3/E, 2008

2. Ming-Hsuan Yang, Narendra Ahuja, Face detection and gesture recognition for human- computer interaction OReilly, 2006.

3. Gary Bradski, Adrian Kaehler, Learning OpenCV: Computer Vision with the OpenCV Library, Shroff/O'Reilly, 2008

4. Stanley B. Lippman, Josee Lajoie, A C++ Primer, Addison-Wesley, 3rd Edition 2008

5. Ivor Horton, Ivor Horton's Beginning Visual C++ 2008, Wiley India Pvt. Ltd/Wrox, 2008

6. Jon Skeet, C# in Depth, Manning Publication, Second Edition, 2008

7. Kogent Solutions Inc. , .NET 3.5 Programming Black Book: Covering .NET Framework, VB 2008, C# 2008, And ASP.NET 3.5 ,
Wiley India Pvt. Ltd, 2010

8. Christian Nagel, Bill Evjan, Jay Glynn, Morgan Skinner, Karli Watson, Professional C# 2008, Wiley, 2008

9. Chensheng Wang, Fei Wang A Knowledge-based Strategy for Object Recognition and Reconstruction, 2009

10. Soo Chahn Lee, Duck Hoon Kim, IlDongYun andSangUk Lee School of EECS, How Can We Evaluate Object Recognition
Algorithms ,IEEE, 25 June 2005

11. M. A. Moni, A B M Shawkat Ali, HMM based Hand Gesture Recognition: A Review on Techniques and Approaches, IEEE, 03
January 2006

12. Jacinto Nascimento, Jorge S. Marques,New Performance Evaluation Metrics for Object Detection Algorithms, ISR/IST, 2006.

13. http://opencv.willowgarage.com/wiki/

14. http://en.wikipedia.org/wiki/OpenCV

15. http://www.emgu.com/wiki/index.php/OpenCV

16. http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html

17. http://msdn.microsoft.com/en-us/library/aa139615.aspx

18. http://www.youtube.com/user/badrepent

19. http://stackoverflow.com/questions/3652327/choice-between-win32-apis-and-net-framework

20. http://stackoverflow.com/questions/3762081/using-c-for-real-time-applications

21. http://stackoverflow.com/questions/3691198/selecting-an-appropriate-ide
22. http://note.sonots.com/SciSoftware/haartraining.html

10