Hand-Eye Coordination using

KSOM
Laxmidhar Behera
Department of Electrical Engineering
Indian Institute of Technology Kanpur
 Visual Manipulation in a Learning Enviroment

•Model based visual servoing requires instantaneous updates

of Image Jacobian (L)
•L depends on visual features, Cartesian depth information of
the object and end-effector and camera transformation matrix

Behera and Nandagopal, IEEE control systems 1999, Patchaikani and Behera Elsevier-RA 2010, Kumar , Patchaikani, Dutta and Behera
Robotica 2008, Kumar, Patel and Behera , Neural Processing Letters 2008, Kar and Behera , Intelligent Service Robotics 2010
SOM Based Control: Redundant Solutions

Swagat Kumar, Premkumar P., Ashish Dutta and Laxmidhar Behera, Visual Motor Control of a 7DOF Redundant
Manipulator using Redundancy Preserving Learning Network, Robotica, Vol 28, No 6, 2010
 Online adaptation algorithm

The online adaptation algorithm is derived

based on the Lyapunov stability condition

• The Lypunov candidate function

• The derivative of the Lypunov

candidate function

• The derivative of the Lyapunov candidate function will be negative definite

Indrazno Sirajuddin, L Behera, TM McGinnity, S Coleman, IEEE Trans Mechatronics, 2014

KSOM based System Identification
 Weight Update

Unsupervised Learning

Supervised Learning
Hand-Eye Coordination
Hand-eye coordination is a process by which biological organism
manipulate objects of interests.

The great football player Diego Maradona had the amazing ability to pass the ball with
accuracy; he had the almost perfect control of the ball as he dribbled past multiple
opposing players on a run; and his amazing reaction time to score a goal given an
opportunity astounded every one. The key here is again the eye-leg coordination - the
precise visual control that he exerted.
Eagles spot their potential prey from a very long distance and catch the
prey with amazing visual control of their speed.

The sand artist Mr Patel depicts the pastimes of Lord Krishna on the sand
by dexterous manipulation of his fingers.

Such hand eye coordinations are visible when one plays computer games,
instrument, and when one is typing or cooking.
 Forward Kinematics

Inverse Kinematics
 EKSOM Model to Learn Inverse Kinematics

Forward Kinematics:

Inverse Kinematics:

Linearized Inverse Kinematics:

Collective Neuronal Response around the winner

Neuronal Response of a single neuron
 The Cost Function for Update of

Coarse Action

Fine Action

Online Training Pair

Cost function for the update of
Weight Update
Parameter Selection
etai=1;
sigmai=2.0;
etaf=0.005;
epsi=0.8;
epsf=0.05;
sigmaf=0.1;
sigma1i=1.0;
sigma1f=0.05;

=[1, 0.005]
Assignment

Select a EKSOM of size 12x7x4

Hand Eye Coordination - Summary
Given any robotic arm, this
lecture presented a KSOM
based network architecture
and learning algorithm by
which a robot can reach any
arbitrary target position
within its own workspace.
The learning algorithm
generates data while actuating
coarse and fine action for the
robotic arm.
The innovative generation of the cost function is
something that need to reflect upon.

L. Behera and Nandagopal K., A hybrid neural control scheme Indrani Kar and Laxmidhar Behera"Visual Motor Control of
for visuo-motor coordination, IEEE Control System Magazine, DOF Robot Manipulator Using a Fuzzy SOM Network",
vol. 19, no. 4, August 1999, pp. 34-41 Intelligent Service Robotics, Vol 3, No 1, pp 49-60, 2010

Swagat Kumar, Premkumar P., Ashish Dutta and Laxmidhar

Nimit Kumar and Laxmidhar Behera, Visual Motor Behera, Visual Motor Control of a 7DOF Redundant
Coordination Using a Quantum Clustering Based Neural Manipulator using Redundancy Preserving Learning Network
Control Scheme, Neural Processing Letters, Volume 20: 11-22, Robotica, Volume 28, issue 06, pp. 795-810, Oct 2010
2004.