FINAL REPORT

Project title: Design and implementation of a gesture controlled robotic arm

Team members:
Harvish Kamlesh Sonar

2012A4PS079P

Sahib Singh Dhanjal

2012A4PS289P

Taizoon Chunawala

2012A4PS298P

Course: ME G511 Mechanisms & Robotics

Semester: First semester 2014-15
Date: 07/11/2014

Abstract
The aim of this project is to explain the designing, fabrication and programming a
prototype of a gesture controlled robotic arm. The arm is controlled by hand
gestures which are recognized using a gesture module kept on the hand of the user,
comprising of an IMU (inertial measurement unit). The project also explores various
applications of gesture controlling. It aims to apply the knowledge of DH notation and
transformation matrices by integrating it in the programme for robot control. The
robotic arm is a TRR type arm with three degrees of freedom and an approximately
semi-spherical workspace. The programming was done on an arduino platform and
an ATmega 382 controller board is used.

Table of contents

The home position

1.

Introduction

Our aim in this project was to design, fabricate and program a prototype of a
gesture controlled robotic arm. The function of a robotic arm is to perform tasks
similar to a human arm, which are too dangerous, difficult or boring for a human. The
robotic arms may be programmed to perform a series of tasks repeatedly or move
according to a set of commands given by remote control.
A gesture controlled robotic arm reads a set of commands by the gestures of the
user. These types of controllers may use a sensor like accelerometer, gyroscopes or
IMU (inertial measurement unit) mounted on the hand of the user, often by a glove,
or may use cameras to recognize the gestures and move accordingly. The gesture
control technology is mostly used in areas where the actions performed are not
always repetitive and a great amount of precision is required mostly used in pick and
place applications. The areas in which this can be applied are in medical field for
performing operations, in nuclear field to handle harmful radioactive materials, and in

extra-terrestrial rovers to collect samples. These can also be used to magnify

gestures by modifying the sensor values for increasing the region of influence or
similar other uses. Hence, there are countless applications and scope is immense.
The prototype is a TRR type robotic arm with 3 degrees of freedom and a 3-D
workspace covering a 25cm radius hollow hemisphere. The user would simply be
mounting an IMU sensor on his forehand and the robotic arm will replicate the
motion of his arm with the help of an arduino code and 3 actuators namely servo
motors.
The prototype is not wireless, however wireless technology can be applied in the
actual product.

2.

Methodology

The IMU placed on the forearm senses the acceleration of the human arm. It senses
6 accelerations, namely angular accelerations for roll, pitch and yaw and linear
accelerations in the x, y and z directions. The three values corresponding to the
accelerations in the x, y and z directions are taken as an input from the IMU and are
integrated to get the displacement of the human arm in the three directions.
These 3 values are then fed to the arduino code appropriately. Different
combinations of these 3 values correspond to the 3 elements of the last column of
the transformation matrix which is computed with the help of Denavit-Hartenberg
convention. The 3 equations thus obtained are solved to get the variables 1, 2, 3,
et cetera through which the actuator servos need to be rotated, subsequently
rotating the servos accordingly to make the robotic arm reach the required point.

3.

Sensors and Actuators

A robot requires sensors to perceive changes in its environment and actuators to

respond to the changes. Following is the list of sensors and actuators with their
respective functions we have used in our project:
a. Inertial Measurement Unit(IMU)

The chief sensor in this project is an Inertial Measurement Unit sensor (IMU).
The basic function of the IMU is to sense 3 variables from the forearm as the
user moves his hand, namely acceleration in the x, y and z direction of the
arm which are the only external inputs for the robotic arm. The sensor
measures these with the help of accelerometers.
b. Arduino UNO (ATmega 328 Controller)
ATmega328 is a single chip micro-controller created by Atmel and belongs to
the megaAVR series. ATmega 328 controller takes input from the IMU and
converts it into angles through which the servos need to be rotated. It is the
only controller on the robotic arm.
The controller takes input of the 3 variables from the IMU sensor and solves the 3
equations which are obtained from the comparison of elements of the transformation
matrix, thus giving an output in the form of angles through which the three servos
need to be rotated.
c. The robotic arm is TRR type and hence uses three servos, one for each
revolute joint and one for the orientation of the end-effector. The servos were
selected based on the amount of torque they would have to counter coming
from the weight of the various components succeeding the actuator when
viewed from the base frame. Torque which the actuators needed to counter
mostly came from the weight of the servos at the distal ends of the links since
the links were to be manufactured using acrylic which is much lighter. The
servo with a nearest higher torque rating was selected as it would ensure
actuation in case of abnormal friction resulting from manufacturing or other
errors. The three servos were:

4.1 kgcm torque servo for the twist joint (T).

The twist joint has a vertical axis of rotation perpendicular to the ground so it
does not have to counter torques from the weight of components but the inertia.
Hence, we do not need an expensive higher rating servo here.

15 kgcm torque servo for the rotary joint (R).

The first R joint has to counter the weight of the subsequent servo as well as the
links. We calculated the torque that the servo needed to counter to finally decide on
a 15kgcm torque servo.

4.1 kgcm torque servo for the rotary joint (R).

The last R joint from the base frame supports just one link and the end-effector
which does not add up much weight. Servo with a 5 kgcm torque was chosen
since the servos with a relatively lesser highest torque rating were equally
expensive and heavy.

1.6 kgcm torque servo for the end-effector

This servo is used to control the orientation of the end-effector. The servo has
been integrated with the prototype just for exhibition since we were not able to
utilise the three inputs from the IMU that were meant for end-effector orientation.
The end-effectors might need more powerful servos based on the application.
The prototype is not meant to lift anything and hence a smaller servo is sufficient
for demonstration.
These three servos required for the T, R and R joints were procured from ecommerce websites Ebay and Amazon.

4.

Design

Link 2
Link 3

Servo Support

Link 1

Servo motors

Figure 1

The arm was designed keeping in mind its manufacturability. The time available was
limited as were the manufacturing facilities, hence we tried to make it as simple as
possible. There are no redundant parts in the robotic arm. The design was
completed in Solidworks and the fabrication in the workshop.

Major considerations in the design were appropriate support to the actuators and
adequate strength of the links in order to support the actuators. The links were
designed keeping in mind the size of the servos they had to accommodate. The link
lengths and thickness of the material were decided upon according to the availability
and structural requirement of the member.
The base for the arm is made of 15mm thick plywood. Wood in addition to being
sturdy is very easy to machine. There are a variety of options available for joining
two pieces of wood from nails to adhesives. The elevation for the first link was
provided by a wooden block and the motor was mounted upside down on a acrylic
sheet platform with the motor horn attached to the stationary wooden block. Hence,
the motor rotates instead of the horn. This made mounting the motor easier and also
the load on the servo horn was reduced by supporting the platform on the periphery
of the horn by a wooden ply just enough thick to bridge the gap between the acrylic
sheet and the wooden block.
The twist joint platform had to accommodate the twist servo as well as the servo
mounts for the first rotary joint. A size just big enough to accommodate both was
selected. A 12mm thick acrylic sheet was used to make the servo mounts. The links
were made of two identical parts such that one of the parts holds the servo at its
distal end while the other supports it. Appropriate slots were made in the link to fit in
the servo.
Link 3 which is to be rotated by the second R servo is similar to link 2.
5.

Fabrication

The fabrication of the arm was completed in the BITS workshop. The arm was
manufactured using 4mm acrylic sheets. Acrylic sheets were chosen over M.S. or
Aluminium Sheets because they are easier to machine. Acrylic is lighter as
compared to metallic sheets too.are. Also, this being a prototype, all the acrylic has
to support is the weight of the servos which does not require the strength of metals.
The T-joint on which the robotic arm is to be supported is made of cheer-wood block.
This joint was required to be elevated as well as it had to support the fixtures for the
arm. Also it was required to be sturdy on the ground so that the arm does not
wobble. Hence it needed to have measurable dimensions of length, breadth and

height. Wood is easy to machine, available cheap and it also is aesthetic. Acrylic
blocks/billets are not available and cutting metal billets, for instance aluminium, to
the right dimensions on a band saw is not only tedious but the surface finish may
also be compromised.
The arms were cut out from acrylic sheets on the band saw machine with the help of
cut-outs of 2D drawings. Slots were then to be made in the acrylic sheets so that the
servos can be fitted in. The slots were made by nibbling on the inside of the required
outline with a drill and then giving proper dimensional finish using a bastard file. The
servos were fixed to the arms using Allen bolts. The servo shaft was attached to a
horn using screws. The horns were then fixed to the links using screws. The servo
support for the first revolute joint was made out of a 12 mm acrylic sheet, since it
needed to be sturdy enough to hold the weight of the entire arm. Being thicker,
nibbling was not a good solution in this case to cut a slot for the servo. Hence, the
slot was made by making a saw-cut on one side of the slot and then cutting
throughout the profile on the band saw machine. Adhesive was not found to be an
apt solution for fixing the servo support to its base because of the amount of moment
it needed to withstand due to the weight of the arm. Therefore a screw was used to
fasten the support to the base along with the use of adhesive.

6.

Circuitry and Programming

Circuitry:
The major part of the circuit is connected to the Arduino. Input sensed from the
MPU-6050 is first converted to usable format within the MPU and then this is sent as
input to the Arduino. The Arduino then performs calculations based on the values of
the analog input we give it via pins A4 and A5 on the Arduino. The Arduino we are
using is an ATMega-328 based UNO board. The computed values are sent to the
servos which are connected to pins 4-7 on the Arduino. The 4 servos are powered
using a 6V battery. The Arduino is powered using the laptop USB port. The servo is
connected to the 6V header pin, the ground header pin on the shield we made and
the signal pin (4-7) on the Arduino.
The MPU is connected to the Arduino as follows:
ON ARDUINO
5V

ON MPU
VCC

GND
A4
A5

GND
SDA
SCL

The servo demands a lot of power and can drain a battery instantly or cause shortcircuiting; hence we have connected the servo to the battery via capacitors, resistors
and a diode so that there is minimum voltage fluctuation and no damage is caused to
the motors.

Programming:
Programming the robotic arm was the part which came in succession to the circuit
design. The programme for the arm was ingenuously developed based on the
learning in the class room. As mentioned in the introduction, the programme
primarily is refining of the raw values obtained from the IMU sensor and solving the
equations to obtain the solutions for the inverse kinematics.
The transformation matrix for the robotic arm is obtained by the Denavit-Hartenberg
method. The displacement of the hand, because of the gesture, in the 3 directions is
obtained by integrating the values from the IMU accelerometer twice with respect to
time. This gives us the right hand side of the inverse kinematics equation. The
equation equating L.H.S and the R.H.S in the simplest form is fed to the code such
that it gives 1 , 2 and 3 or the angles through which the three servos need to be
rotated so that the robotic arm exactly replicates the gesture.
The computation of forward and inverse kinematics for a TRR arm by DH method
has been shown in Appendix 1 and Appendix 2 respectively.
Since we were using a custom sensor (MPU-6050) hence we required different
libraries so that we could program the bot with sufficient ease. The libraries we used
are:
1.
2.
3.
4.
5.

MPU6050_6Axis_MotionApps20.h
Wire.h
Math.h
Servo.h
I2Cdev.h

The basic algorithm of the program is as follows:

Get the raw readings from the IMU

Convert these raw values to usable form
Using these values as input, establish relation between (x,y,z) position and
the angle of rotation of each servo using inverse kinematic relations.
Feed these computed angles of rotation to the respective servos.
Go back to step 1.

This whole task is done by implementing functions present within the given libraries.
The actual code is given in the Appendix 3

7.

Conclusions and Further Work

Many gesture controlled robots make use of image processing to perceive
changes in the gestures. Image processing is an expensive technique which
requires a certain environment for it to function. While the IMUs are
inexpensive, they are not very accurate. For applications where accuracy is
very important like the medical sector, IMU is not the best option. Their control
is not easy and it takes a finite time for the arm to respond to the gesture
corresponding to the multiple computations. The IMU gives out acceleration
as its output which needs to be integrated to get the displacement, which then
by using inverse kinematics gives us the rotation corresponding to each
actuator.
Controlling the orientation of the end-effector in addition to making it reach the
requisite point is crucial in real-life applications especially when the endeffector has multiple degrees of freedom. Hence, integrating orientation along
with position using one or multiple IMU sensors is important and needs to be
explored. A programme which can accurately convert the accelerations from
the IMU to displacements is vital to achieve proper gesture based response.
Further the robotic arm can be made to operate wirelessly by using an RF
module. This will enhance its ease of use and make it less cumbersome. This
idea can also be used to control the robotic arms installed on extra-terrestrial
rovers, while the operator is on the earth.

The project demonstrates the processes involved in manufacturing the

prototype of a robotic arm controlled by gesture. The ideology used in making
this prototype can be further enhanced to manufacture a real product capable
of lifting weights at the required site. The scale of the arm may be varied
based on the application. But the most common application would be in
industries to lift heavy loads. For this, the links need to be made up of a
suitable material, for instance aluminium or MS, which would have enough
strength to withstand higher bending forces. Also, the servos to be used
should be of higher rating in order to provide enough torque to counter the
deadweight.
The overall applications of this technology are quite widespread into various
fields. It ranges from handling of radioactive substances, where human

involvement is too difficult, to its use in biomedical field to perform

complicated surgery in an area lacking professional surgeons, requiring
utmost precision. Other applications involve use in extra-terrestrial rovers,
underwater trenches and common industrial tasks.
Finally by using proper simulation and modelling, functional workspace can be
maximized, simultaneously avoiding singularities.

References
1. Mittal, R.K., and Nagrath, I.J. Robotics and Control, New Delhi,
McGraw-Hill, 2003
2. http://invensense.com/kr/mems/gyro/documents/articles/Gesturerecogn
itionSUI.pdf
3. https://www.kickstarter.com/projects/ufactory/uarm-put-a-miniatureindustrial-robot-arm-on-your
4. http://www.instructables.com/id/DIY-robot-arm-/
5. http://lukagabric.com/arduino-mpu6050/
6. http://hobbylogs.me.pn/?p=47
7. http://www.ez-robot.com/Community/Forum/Thread?threadId=5131
8. http://nevit.blogspot.in/2014/07/arduino-mpu-6050-pitch-roll-with-2.html

Appendices
Appendix 1