Remote Vision Integrated WiFi-based Robotic Arm

Abstract~ A research was done to design and build a position control

robotic arm with an FPGA. Digital PWM was used to control position
of servo motors to perform pick and place controlled operation, using
FPGA. Integrated remote vision at the robotic arm was implemented
using IMAQ toolkit in LabVIEW. To increase speed and reliability the
joystick was used to control the robot arm from user end. The
communication between the robotic arm to the computer was made
possible by using WiFi router connected to the cRIO. WiFi make it to
work in hazardous areas, reduced manufacturing time in industry.
LabVIEW web publishing tool was used to publish GUI on HTTP for
remote control of robotic arm.
Keywards~cRIO, FPGA, IMAQ, Robotic arm, Remote
vission, web publishing .

I.INTRODUCTION
The term robot comes from the Czech word ,
generally translated as "forced labor." This describes the
majority of robots fairly well. Most robots in the world are
designed for heavy, repetitive manufacturing work. They
handle tasks that are difficult, dangerous or boring to human
beings. The most common manufacturing robot is the
robotic arm. A typical robotic arm is made up of seven
metal segments, joined by six joints. The computer controls
the robot by rotating individual step motors connected to
each joint (some larger arms use hydraulics or pneumatics).
Unlike ordinary motors, step motors move in exact
increments. This allows the computer to move the arm very
precisely, repeating exactly the same movement over and
over again. The robot uses motion sensors to make sure it
moves just the right amount.
An industrial robot with six joints closely resembles a
human arm -- it has the equivalent of a shoulder, an elbow
and a wrist. Typically, the shoulder is mounted to a
stationary base structure rather than to a movable body. This
type of robot has six degrees of freedom, meaning it can
pivot in six different ways. A human arm, by comparison,
has seven degrees of freedom. In present era, Robotic Arm
controller is the challenging and demanding activity in
industrial field as well as military and other applications.

Jidhu Mohan M, Prashant Kumar Tripathi, K.V Gangadharan
Department of Mechanical Engineering,
National Institute of Technology Karnataka,
Surathkal, Mangalore, India
E-mail: jidhu.mohangmail.com
This designed project is the educational based concept.
Embedded system provides solutions to the implementation
of robot control system very easily in digital way. The
introduction of field-programmable gate array (FPGA)
technologies has blurred the distinction between hardware
and software [1]. Traditionally a hardware circuit used to be
configured at manufacturing time. With field programmable
technology, it is possible to configure the gate level
interconnection of hardware circuit after manufacturing [2].
This flexibility opens new application of digital circuits,
subsequently; the FPGA can be reprogrammed to perform
another specific function without changing the underlying
hardware. Thus from a user perspective, a reprogrammable
hardware board can perform a function indistinguishable
from that of a processor. Nevertheless, the programming
mechanisms and the programmer`s view of the hardware are
very different [3].The present paper involves the
Hardware/software design for robot arm control application
which include three main parts controller, NI hardware`s,
Robot arm model with servo motor,WiFi Router, joystick
and remote vision camera. The design composed three
different parts, in which two different physical property
devices combine on one application platform with software
thread. FPGA controller is the software part, which provides
free environment to user interface [4] as in Figure 1[4].cRIO
the hardware part and Robot arm with functional gripper is
the mechanical part of system. All parts are interconnected
with each other to perform the designed algorithmic task to
achieve pick and place operation With each movement the
motor, rotary encoder reads the position of a disk attached to
the motor. This position information is sent back to the
FPGA Controller so it knows how much further the motor
needs to be turned or if the motor has turned too far.

Fig.1.Typical FPGA structure

Jidhu Mohan M, Prashant Kumar Tripathi, K.V Gangadharan
DRDO Sponsored Eighth Control Instrumentation System Conference, CISCON-2011(An International Conference)
197
II. SYSTEM OVERVIEW

The design concept is implemented with the help of robot
arm model. Robot Arm is a 5 axis robotic manipulator each
has two possible orientations consist of a rotating base, three
links and a gripper. The five axes consist of the Shoulder,
elbow, wrist having 180, 300, 60 degree rotational limit
with vertical movement. The axis at base is having 270
degree rotational limit with horizontal movement. The
control signals to the motor define the movement of each
arm in three phases. The NI 9472/9474 provides
connections for eight digital output channels were used to
run this servo motors Figure 2.


Fig.2. NI 9472/9474 Terminal and Pin Assignments.

Servo Motors come with three wires or leads. Two of these
wires are to provide ground and positive supply to the servo
DC motor. The third wire is for the control signal. These
wires of a servo motor are color coded. The red wire is the
DC supply lead and must be connected to a DC voltage
supply in the range of 4.8 V to 6V. The black wire is to
provide ground. The color for the third wire (to provide
control signal) varies for different manufacturers. It can be
yellow (in case of Hitec), white (in case of Futaba), brown
etc. were used with aluminum link Figure 3.




Fig.3. Servo motor used with aluminum link.

Remote vision on robotic arm is developed using wireless
camera includes power adapters, receiver and it ranges up to
30m.Wireless camera is interfaced with Lab VIEW IMAQ
through 'G` programming to capture images and real time
video Figure 4.

Fig. 4. Wireless camera for remote vision.


Fig. 5.WiFi router for robotic arm.

For Wi-Fi control of robotic arm a router is used with
robotic arm mounted on same station to transmit data over
the Wi-Fi to the cRIO (compact reconfigurable input output
device) from a remote location. National Instruments
provides multiple hardware platforms with FPGA
technology. Figure 6 shows the CompactRIO programmable
automation controller with an FPGA on the backplane of the
chassis. NI also provides programmable plug-in FPGA
boards for desktop computers.



Fig.8.NI CompactRIO automation controller
DRDO Sponsored Eighth Control Instrumentation System Conference, CISCON-2011(An International Conference)
198
III. LABVIEW AND FPGA PROGRAMMING
LabVIEW, (Laboratory Virtual Instrumentation Engineering
Workbench) developed by National Instruments, is a
graphical programming environment suited for high level or
system level design. It allows integrating different signal
processing components or subsystems within a graphical
framework. This software package is one of the first
graphical programming products, and is currently used in
academia and industries for data acquisition, remote control,
simulation, and analysis [5]. LabVIEW uses dataflow
programming, where the flow of data determines execution.
LabVIEW incorporates a graphical user interface (GUI)
programming environment to produce programs that mimic
laboratory instruments. These programs are called Virtual
Instruments. Direct Control mode enables the user to control
the static arm and a graphical simulator arm using the
joystick. The flow chart and LabVIEW programme of the
direct control panel is presented in Figure 9.





















Fig.9. Flow chart of direct control panel

The FPGA program has two parts host and target. The target
will be deployed into the cRIO. The host runs in the FPGA
ASIC (application specific integrated chip) in the cRIO. G-
programming is shown in Figure 10,12 & 13 . Here the
PWM generation program was deployed into FPGA. The
LabVIEW program was converted to the VHDL code and
deployed into the Xilinx by the inbuilt compiler. After the
compilation the target was opened in the host and the PWM
pulse width is changed for the rotation of the servos.
At host the program was made in such a way that the control
robotic arm can be done in two ways
(i) Joystick Control
(ii) Keyboard control
The operator has a flexibility to select any of the control to
define profile path of robotic arm.Future work is to give
user best path using fuzzy logic.

Fig.10.Target VI G programming

The key press was detected in the program using the data
connectivity tool kit in the LabVIEW. The keys were
assigned for the servos. So if the desired key press was
detected, the corresponding servo angle was changed in a
controlled manner block diagram in Figure 11.


Fig.11. Key Press Detection G programming

Rare motions like the rotation of the end effector were
controlled by multiple key presses.
Robot Station
D-Link cRIO Robot
Arm
Remote vision
User
Joystick/controller
Simulator/GUI
Wi-Fi
DRDO Sponsored Eighth Control Instrumentation System Conference, CISCON-2011(An International Conference)
199

Fig.12. Servo angle to pulse width conversion


Fig.13. Incrementing servo angle for the key press

The real time image and Video acquisition was made
possible using the LabVIEW IMAQ in Figure 11. The
Video was stored in system and image processing was
done in LabVIEW.


Fig.14. IMAQ Image acquisition

IV. CONCLUSION

Robot arm control in nuclear power plants is an active
research area, covering many different fields of science and
engineering such as artificial intelligence, interfacing with
computer, converting signals (A/D), machine learning, and
kinematics. However, when cleaning of nuclear plants and
replacing moderators in nuclear plants, intrinsic difficulties
like health hazards for human being and system shutdown
due to replacement work occurs. This paper presents the
LabVIEW based data acquisition card system to control the
robot arm mimics the human arm. The 5-joint mechanical
arm which is built from scratch; a hardware robot arm was
completely and successfully implemented and linked them
using the LabVIEW & cRIO DAQ.The interface developed
is powerful tool for controlling the robotic arm. Software
has been designed in such a way to make it more user-
friendly and understandable. It has been tested in many
areas and competitions. Its performance was observed to be
excellent. The GUI is published on HTML and the robotic
arm can be controlled remotely throught the controls at user
end. Just as it was an educating experience conducting this
project, its outcome, can be used as an educational tool for
introducing the LabVIEW based data acquisition card real-
time applications.

REFERENCES

[1] Ying-Shieh Kung, Gua-Shieh Shu, FPGA-based
Motion Control IC for Robot Arm, IEEE-2005, pp
1397-140.
[2] Swagat Kumar, Amit Shuklay, Ashish Duttaz and
Laxmidhar Beherax, Technique for Visual Motor
Coordination of a 6 DOF robot manipulator,
Intelligent Control Part of IEEE Multi-conference
on Systems and Control 1-3 October 2007, pp
544-549.
[3] S. Kawahito, CMOS Sensors for Smart and
Advanced Imaging, the 3rd Korean-Japanese
Student Workshop, Oct.2007, pp 6-15.
[4] Sung Su Kim and Seul Jung, Hardware
Implementation of a Real Time Neural Network
Controller with a DSP and an FPGA, Proceedings
of the IEEE International Conference on Robotics
and Automation, New Orleans, LA. 2004, pp 4739-
4644.
[5] Korrapati R.; Anderson, J.A; Swain, N.K; Swain
M; System Modeling Using Virtual Instruments
Southeast Con, 2002.Proceedings IEEE Digital
ObjectIdentifier:10.1109/ .2002.995570Publication
Year: 2002, Page(s): 121 - 126.
[6] Norman S. Nise.Control Systems Engineering,
4th edition. 2004, John Wily & Sons, Inc
[7] The Canandarm: Remote Manipulator System
(RMS),Available:http://ffden2.phys.uaf.edu/211.w
eb.stuff/Adamczak/rms.html.
[8] G. Hirzinger, N. Sporer, M. Schedl, J. Butterfass,
M. Grebenstein, Robotics and Mechatronics in
aerospace, The 7th International Workshop on
Advanced Motion Control, 2002, pp. 19-27.
DRDO Sponsored Eighth Control Instrumentation System Conference, CISCON-2011(An International Conference)
200
[9] Y. Woo-Keun, T. Goshozono, H. Kawabe, M.
Kinami, Y. Tsumaki, M. Uchiyama, M. Oda, and
T. Doi, Model-based space robot teleoperation of
ets-vii manipulator, IEEE Trans. on Robotics and
Automation, vol. 20, no. 3, pp. 602-612, 2004.
[10] T. Tayh-Jong, A. Bejczy, G. Chuanfan, and X.
Ning, Intelligent planning and control for
telerobotic operations, Proc. of IEEE/RSJ Int.
Conf. Intelligent Robots and Systems, pp. 389-
396, 1994.
[11] J. Park and O. Khatib, A haptic teleoperation
approach based on contact force control,
International Journal of Robotics Research, vol. 25,
no. 5-6, pp. 575-591, 2006.
[12] T. Asfour and R. Dillmann, Human-like motion
of a HumanoidRobot Arm Based on a Closed-Form
Solution of the InverseKinematics Problem,
Industrial Application of Informatics
andMicrosystems (IAIM), Computer science
Department, University of Karlsruhe, Germany.
[13] John. J. Craig, Introduction to Robotics,
Mechanics and Control, 2nd Edition, Addison-
Wesley, 1989.
[14] National Instruments, cRIO-9022, NI-9472 and
NI- 9427, user manual: January 1999 Edition.
[15] D. F. Styer, The strange world of quantum
mechanics ~cambridge U.P., New York, 2000.





Fig.6.Front Panel for robotic arm control


Fig.7.Robotic station with robot arm
DRDO Sponsored Eighth Control Instrumentation System Conference, CISCON-2011(An International Conference)
201