USB GESTURE DEVICE

Chapter 1
INTRODUCTION

ABOUT USB GESTURE DEVICE

Mister Gloves is a wireless USB gesture input system that enables a person to use a
computer by performing intuitive hand and finger motions in the air. While wearing a glove
controller on the right hand, the user can move the cursor by forming a pointing gesture and
click by curling the index finger or thumb. Other right hand gestures enable scrolling and
provide access to various keyboard shortcuts.

BASE DESIGN

A simple cotton glove is fitted with an accelerometer. This glove is then worn by the
user and the desired actions are performed as shown in . The accelerometer senses the
orientation of the hand in space through an inbuilt capacitive system and feeds out electrical
signals as unique voltages for each unique orientation. These voltages are converted to digital
values by the Microcontroller and then sent to the wireless XBEe transmitter-receiver pair,
which passes on this data to the computer where it is manipulated by our program running in
the background

ACCELEROMETER

The accelerometer is the device which will generate corresponding electrical signals
to control the on-screen elements. The accelerometer used here is the 3-Axis Accelerometer
with an easy analog interface and running at a supply voltage of 3.3V, which makes it ideal
for handheld battery powered Electronics. The accelerometer will experience acceleration in
the range of +1g to -1g as the device is tilted from -90 degrees to +90 degrees. In order to
determine the angle of tilt, θ, the A/D values from the Accelerometer are sampled by the
ADC channel on the microcontroller. The acceleration is compared to the zero-g offset to
determine if it is a positive or negative acceleration. This value is then passed to the tilt
algorithm. When applied to all three axis, we are able to calculate the orientation of hand in
three dimensional space.
Dept of CS & E, SIT, Mangalore Page 1
USB GESTURE DEVICE

The serial data sent by the microcontroller via the UART is read by the software
written in Visual Basic using the MSCOMM32.OCX module which interacts with the
System Kernel However one may use any system programming language that can provide
access to the core kernel libraries to interact with the mouse parameters. The following
settings are required for initialization:
1. Defining the Port no.
2. Setting up the baud rate (9600 bps is the default), setting up parity bits and other
error correction parameters
The following are the essential functions that would be needed to be reused as modules:
1. Constructor for the Mouse Pointer Initialization.
2. State functions for defining the mouse clicks and the associated events
3. Function for sending real time processed of the X, Y positional parameters to the
kernel libraries

Dept of CS & E, SIT, Mangalore Page 2

USB GESTURE DEVICE

Chapter 2

SYSTEM ANALYSIS
2.1 IDENTIFICATION OF THE NEED
Communicating with computer involves use of touch screens, wireless/wired mouse along
with keyboards.“Human Computer Interfacing Device” mapping Hand Gestures, most
intuitive communication gesture, to communicate with computers, Developments in field of
communication have enabled computer commands being executed using hand gestures. This
proposed novel solution achieves effective interaction with Internet giving quality feedback,
intuitive control with complete mobility. Simple inertial navigation sensor like an
accelerometer is utilized to get dynamic/static profile of movement to navigate the mouse or
gyroscope to rotate 3-D virtual objects. Accelerometer profiles are converted into wireless
interactivity. The device involves non-tactile interaction with computer screen where mouse
pointer reacts to and manipulates screen content in accordance with hand gestures. The
applications envisioned: interaction using gesture technology for effective communication
empowering physically challenged to interact with computers including 3-D graphic
interactions
Our gesture input system can be conveniently used by anyone who wishes not to be tied
down to a desk when using a computer, making it perfect for giving presentations or web
surfing from the couch. The intuitive hand motion controls also allow it to serve as an
alternative video game controller. Additionally, since our input system does not exert
pressure on the median nerve at the wrist while in use, it may prevent the development of
carpal tunnel syndrome and other repetitive stress injuries.

2.2 PRELIMINARY INVESTIGATION

In the current era, computers are being integrated into every aspect of our lives making
it essential to move away from the conventional keyboard/mouse interface and delve into
intuitive methods of interacting with the computers and other appliances around us. Also,
such devices being tools designed to aid humans, it is in our best interests if we can make
them adapt to our natural communication patterns rather than otherwise.

Dept of CS & E, SIT, Mangalore Page 3

USB GESTURE DEVICE

Human hand gestures are a means of nonverbal interaction among people. They range
from simple actions of pointing at objects and moving them around to the more complex
ones that express our feelings or allow us to communicate with others. To exploit gestures in
Human Computer Interfacing, it is necessary to provide the means by which they can be
interpreted by computers. Though various efforts are being made in this direction they face
issues of variable reliability, cost and convenience and often need to provide samples to train
the device. There was an effort in this direction which eventually lead to filing a patent a
careful study reveals that this device is inadequate for 3-D interactions. The inertial sensor,
the accelerometer along with a microcontroller, maps the orientation and position of the hand
in 3-D space. This data is then transmitted to the end application where it can be interpreted
and acted on, thereby simulating wireless interactivity.[1]
[ Acceleglove by Anthrotronix http://www.acceleglove.com/] istead u put [1 say]this you do it at
the end
The inspiration to develop such a device is manifold. Firstly, it aims at venturing into
the field of physiological computing and permitting convenient interaction with the
surrounding appliances through minimal hardware, as compared to complicated magnetic
flux sensors or ultrasonic. Also, the use of hand gestures for interaction avoids the most
prevalent injury due to continuous use of the keyboard and the mouse, the Carpal Tunnel
Syndrome which occurs when the median nerve that runs from the forearm into the Hand
gets pressed at the wrist. For the users suffering from Rheumatoid Arthritis causing loss of
function at the finger joints it would be easier to use simple hand gestures which don’t
require the use of fingers but only hand.[2]
[Microsoft Surface http://www.microsoft.com/SURFACE/Default.aspx]
Movements to perform tasks on a computer. This technology is also of prime
importance in dealing with 3-dimensional objects, effective teaching aid and for providing an
alternative to the traditional joysticks in the gaming industry.

2.3 SCOPE OF WORK

Our input system, while somewhat bulky, is extremely easy to use once it is worn. The
right hand mouse controller is precise enough to highlight specific text as well as click on the
small buttons in the taskbar. With a steady hand, the cursor can be moved a single pixel at a
time. The slight delay added after the mouse button clicks assists successfully in double-
clicking operations. Switching between different gestures on the right hand controller is also

Dept of CS & E, SIT, Mangalore Page 4

USB GESTURE DEVICE

seamless. For example, the right hand can quickly traverse from controlling the mouse cursor
to controlling mouse scrolling.

Dept of CS & E, SIT, Mangalore Page 5

USB GESTURE DEVICE

Chapter 3
SOFTWARE REQUIREMENT SPECIFICATION

3.1 INTRODUCTION
3.1.1 Purpose
The main objective of this document is to illustrate the requirements of the project.
USB Gesture Device The document gives the detailed description of the both functional and
non functional requirements.

3.1.2 Scope
This proposed multi-functional portable device for better human – computer interaction using
hand gestures can be applied in the following applications:
• Replace the mouse as a more convenient and natural interaction peripheral.
• Interacting with 3D objects on computer screen
• Easy control of Robots, Robotic Arms and Human Controlled Automation
• Easy Home Automation
• Effective Teaching / Animation / Design Aid
• Control of Machines for High-Precision Jobs
• Easy accessibility tool for people with disabilities
• When used with other inertial sensors the glove can be used to manipulate objects in 3
dimensions.
• Control over distant actuators connected via a PC interface.
• It can be used extensively in the gaming industry for remote location manipulation

3.2 OVERALL DESCRIPTIONS

3.2.1 Product Perspective
The human wears the gloves that are fitted with the accelerometer module and the micro
switches in the right. This means that mere movements of hand (which need to be
predefined) will translate into corresponding interaction with on-screen elements.

Dept of CS & E, SIT, Mangalore Page 6

USB GESTURE DEVICE

3.2.2 User classes and Characteristics.

USER: The User Wears the Gloves and Perform Various Hand Movements in order to carry
out the function that are similar to that of a mouse.
3.2.3 Product Activity
1. Navigate through various Microsoft Windows menus.
2. Drag a window to a different location on the screen.
3. Move the cursor on screen
4. Interact with on-screen elements.
5. Maximize and minimize windows.
6. Interact with 3-D objects/images on screen
7. Gaming
3.2.4 Assumptions and dependencies
• Only One User can use at a time
• LAN or Wireless LAN connection is a must. (give why you need to have
LAN/Browser) I feel these may be removed as it is networking project
• Proper browsers should be installed.

3.3 SPECIFIC REQUIREMENTS

This section describes the functional Requirement of the System.
3.3.1 Hardware Requirements
• Any processor above 2.4 GHz
• Hard Disk : 10 GB Or Above
• RAM : 512 MB Or Above
3.3.2 Software Requirements
• Operating System : Windows 2000/Windows 2003/Windows XP
• Development Language: Visual Basic.
3.3.2.1 Microsoft Visual Basic 2008

Microsoft Visual Basic 2008 is an evolution of the Visual Basic language that is
engineered for productively building type-safe and object-oriented applications. This
generation of Visual Basic continues the tradition of giving you a fast and easy way to

Dept of CS & E, SIT, Mangalore Page 7

USB GESTURE DEVICE

create .NET Framework-based applications. Visual Basic allows developers to target

Windows, Web, and mobile devices.

3.3.2.2 HTML

Hyper Text Markup Language is the predominant markup language for web pages.
It provides a means to describe the structure of text-based information in a document — by
denoting certain text as links, headings, paragraphs, lists, and so on — and to supplement that
text with interactive forms, embedded images, and other objects. HTML is written in the
form of tags, surrounded by angle brackets. HTML can also describe, to some degree, the
appearance and semantics of a document, and can include embedded scripting language code
(such as JavaScript) which can affect the behavior of Web browsers and other HTML
processors.

3.3.3 Functional requirements

This subsection describes the various processes provided by the system. Functional
requirements specify which outputs should be produced from the given inputs. They describe
the relationship between the input and output of the system. For each functional requirement,
a detailed description of all the data input and their source, the units of measure and range of
valid inputs are given.
3.3.3.1 Click & Positioning
Introduction: This will point the cursor and click on the item
Input: Curling the index finger performs a left click while thumbing down performs a right
click
Output: Cursor points to an item and open it.
Processing: Signals will be sent from the gloves to the MCU.
3.3.3.2 Scrolling
Introduction: To Scroll the Page
Input: In the right hand “downward fist” position, moving the hand upwards scrolls up and
moving the hand downwards scrolls down.
Output: Movement of the Page.

Dept of CS & E, SIT, Mangalore Page 8

USB GESTURE DEVICE

3.4 PROCESS MODEL

In developing this project, iterative model approach is used. In which the
development of system is done in phases. The different phases area as follows.

A. Getting data from the Serial port

The data is sent in packets, each packet representing one single positional state. Each
packet is 32 bits in size – consisting of 8 bits each of X, Y, Z and Button State information.
The values of X,Y and Z range from 0 to 255 which are mapped from the voltage from the
accelerometer.

B. Calibrating
Since the values of acceleration may vary with altitude as well as with various people
using it. The displacement of the positional parameters due to unintentional motions is to be
neutralized. For this, we take the first 50 set of packets as sampling packets – to these packets
we separate out the individual values and take an average, the person wearing the glove is
expected to rest his hands in the rest position. The normal value hence obtained is used for
subsequent mapping of co-ordinates into proportionate displacement.

C. Error Detection
The reasons for errors can be –
• Signal noise
• Intermittent connection at the hardware
• Packet loss
D. Processing Gestures
The raw coordinates obtained after the error correction are used in identifying
gestures which are pre-defined. A button press on the glove puts it into the Gesture Mode;
when in this mode, the co-ordinates are passed onto the Gesture Functions which constantly
monitor and store previous co-ordinate data to recognize a pattern. These patterns can be
customized and are to be pre-defined as required.

Dept of CS & E, SIT, Mangalore Page 9

USB GESTURE DEVICE

3.5 DEVELOPMENT APPROACH

Here top down approach followed by Bottom-Up Approach is used. A top-down
approach is essentially the breaking down of a system to gain insight into its compositional
sub-systems. In a top-down approach an overview of the system is first formulated,
specifying but not detailing any first-level subsystems. Each subsystem is then refined in yet
greater detail, sometimes in many additional subsystem levels, until the entire specification is
reduced to base elements.
A bottom-up approach is the combining different modules together of systems to
give rise to final systems, thus making the original systems sub-systems of the emergent
system. In a bottom-up approach the individual base elements of the system are first
specified in great detail. These elements are then linked together to form larger subsystems,
which then in turn are linked, sometimes in many levels, until a complete top-level system is
formed.
First we are connecting Atmel ATmega16 Microcontroller & all other basic
components to the printed Circuit Board, using Soldering Machine and Lead. Then we write
the code for the glove controller and Burn the code in Chip. Then we fix the chip in the PCB.
The entire PCB is then fitted in the glove for proper functioning. The user wears the glove
and perform hand movement for cursor to move.

3.6 PERFORMANCE REQUIREMENTS

A performance requirement of a software requirement specification specifies the
performance constraint on the software system.
 Static requirements:
It is a single user system. It includes modules and a number of files. It can operate on
any OS.

Dept of CS & E, SIT, Mangalore Page 10