San Diego State University College of Engineering

A Web-Based Mobile Robotic System for Control

and Sensor Fusion Studies

1 2 3 1
Christopher Paolini , Gerold Huber , Quentin Collier and Gordon K. Lee

1Deptof Elect &Comp Engr 2Management Center 3IUTde Bethune

5500 Campanile Drive Innsbruck Networks & Telecomm Dept.
San Diego State University University of Applied 62408 Bethune, France
San Diego, CA 92182 Sciences
Universitätsstraße 15
6020 Innsbruck, Austria
San Diego State University College of Engineering

Outline of Presentation:

➢ The Mobile Robotic System Overview

➢ The ANFIS Algorithm

➢ Sensor Integration

➢ Graphics User-Interface

➢ Results

➢ Conclusions and Future Work

San Diego State University College of Engineering

Goal:

Develop a mobile robotic testbed to investigate

several control algorithms and sensor fusion
techniques

Approach:
Use web-based video streaming and embedded
control architecture for flexibility and robustness
San Diego State University College of Engineering
 iRobot Create® Platform
• Single board computer
(SBC)
• Voyage Linux (Debian)
• Unibrain Fire-i™ digital
camera (IEEE 1394)
• Proxima 802.11g PCMCIA
adapter card with external
5db gain antenna
• Single Board Computer (SBC)
• Linux Voyage
• Unibrain Fire-i™ Digital Camera
• Linksys Wireless-G PC adapter
card
• iRobot Create® platform
San Diego State University College of Engineering

iRobot Create® with Sensor Arrays

 Experimental iRobot developed at San Diego State
University

• iRobot Create® based robot designed with several sensors

Unibrain Fire-i™ Digital Camera

• Streaming video
• Web teleoperation 2.5dBi gain indoor omni- IR Sensor

• ANFIS automation directional antenna

Thermal Sensor

Arduino Mega MCU

9DOF Inertial Measurement

Unit

Migrus C787 DCF-P single

board computer with a 1.2GHz
Eden ULV Processor
360° Ultrasonic Sensor Array

802.11g PCMCIA
Transceiver
San Diego State University College of Engineering

Sensors

3-Axis Rate Gyroscope 3-Axis Accelerometer 3-Axis Magnetometer

Inertial Measurement Unit Controller

San Diego State University College of Engineering

Ultrasonic sensor array using an Arduino Mega 2560 microcontroller

 San Diego State University College of Engineering

Ultrasonic Sensor Array

MaxBotix LV-EZ1 Ultrasonic Sensor

An array of 10 MaxBotix LV-EZ1 sensors Each MaxBotix sensor provides a 36 degree FOV.
suspended on two circular plates.
San Diego State University College of Engineering

Thermal Sensor
 How the iRobot Adjusts Its Heading
 The iRobot changes its azimuth by sending a 16 bit
signed value in the range [-2000, 2000] mm that
defines a turning radius
 The turning radius is a ray from the center of the
turning circle to the center of the robot
 r > 0  robot turns left
 r < 0  robot turns right
 Special cases: r = 32768 or 32767 (0x8000 or
0x7FFF) causes robot to move straight
r1 r2  r = 0xFFFF  robot turns in place clockwise
 r = 0x0001  robot turns in place counter-clockwise

Large radius  small curvature

Small radius  large curvature
How the Web GUI Determines the Turning Radius
 x
 =  tan −1 
(x,y)
 y 
In quadrant I and IV for >5

II I
r = −23.8 − 2142.79
 In quadrant II and III for <-5

r = −23.8 + 2142.79
III IV

Assume r varies linearly

with 
 Inertial Measurement Unit (IMU)
• Robot has an onboard 9DOF IMU
• Incorporates four sensors: LY530ALH single-axis
gyro, LPR530AL dual-axis gyro, ADXL345 triple-
axis accelerometer, and a HMC5843 triple-axis
magnetometer
• Gives nine degrees of inertial measurement
• LY530ALH: STMicroelectronics ±300 °/s analog
yaw-rate gyroscope
• HMC5843: Honeywell HMC5843, a 3-axis digital
magnetometer outputs Euler X,Y,Z orientation vectors
and roll () and pitch () angles (tilt sensor) with 12- xh = x cos  + y sin  sin  − z cos  sin 
bit ADC at 10 Hz yh = y cos  + z sin 
y
• MCU computes azimuth or “yaw” with an accuracy of Az = tan −1 h
xh
1-2 º
 How to Model Robot Dynamics while Turning?
• We want the absolute bearing defined by the
remote user with the GUI to equal the absolute
bearing reported by the IMU
• How to compensate for unknown system
dynamics?
• Can define a neural network to model robot (plant)
dynamics and use training data to tune network
parameters
Define a set of 4-tuple training data:  d  d  a  a
i i i i
•
which are the ith desired (from GUI) azimuth,
azimuth rate, actual (from IMU) azimuth and  d1  d1  a1  a1 
azimuth rate, respectfully.  2 2
 d  d  a  a 
2 2

 
From the HMC5843 digital
magnetometer  n n
 d  d  a  a 
n n
From the LY530ALH yaw
rate sensor
 San Diego State University College of Engineering

ADAPTIVE AJAX-BASED STREAMING VIDEO SYSTEM

AJAX based Web interface for

telerobotic control of the iRobot Create Encoding bit rate as a function of fps
Virtual Force Field Approach
for Obstacle Avoidance
 The Certainty Grid

• 2-Dimensional array of cells

• Each cell contains a Certainty Value (CV)
• CV indicates the measure of confidence that an object
exist within a cell
• Instantaneous map for obstacle representation
• dx = dy = 15 cm
• 21 by 21 square cells represent the Certainty Grid

Front
 Method for Updating Certainty Values

Each sensor corresponds to a particular angle Ө,

based on its position on the sensor assembly
(1)
At a given time, a sensor returns a distance d

Eq. 1 and 2 transform (d, Ө) → (x’,y’)

(2)
Obstacle Avoidance for Path Planning Task

Port Side
Front
San Diego State University College of Engineering

Cells Located on the Acoustic Axis

San Diego State University College of Engineering

(a) Histogram Grid ; (b) Snapshot of Video Camera

San Diego State University College of Engineering

The ANFIS Architecture

layer 1
layer 4

layer 2 layer 3
A1 xy layer 5

x
 
w1 w1
A2
w 1f 1

f

 
w2
B1 w 2f 2
w2

y
xy
B2

If x is Ai and y is Bj, then f i = p i x + q i y + ri

nr
f=  w ifi
i =1
San Diego State University College of Engineering

Off-line Training

Forward pass: consequent parameters

X i+1 = Xi + Si+1 a i+1 (b T

i+1 − a T
i +1X i )


Sia i+1a T S 
Si+1 = Si − i+1 i , i = 0, 1, ..., P − 1
T 
1 + a i +1Sia i +1 
Backwards pass: premise parameters
P P P
2 2
E =  Ei =  (y di − y i ) =  ei
i=1 i=1 i =1
San Diego State University College of Engineering

On-line Learning

Forward pass: consequent parameters

Backwards pass: premise parameters

San Diego State University College of Engineering
San Diego State University College of Engineering

Simulation Results
 System Integration through an Arduino MCU

• Multiple sensors: thermal (person/fixture differentiation), ultrasonic

(collision avoidance and path planning), IR (automatic docking),
video (telerobotic control), and magnetometer and accelerometer
(orientation and position) have been tested and integrated into the
overall system architecture.

 ANFIS controller is being implemented on an Arduino Mega that

uses the magnetometer and accelerometer output for path tracking.
 Initial simulation studies have been conducted using a MISO
controller to test this proof of concept.
 Then performed actual experimentation using one and two inputs to
the ANFIS controller
Initial Web Browser Interface
San Diego State University College of Engineering
San Diego State University College of Engineering

Experimental Results

Bearing Scenario
San Diego State University College of Engineering

Bearing Scenario
San Diego State University College of Engineering

Constant Turn
San Diego State University College of Engineering

Following Scenario
San Diego State University College of Engineering

Learning Scenario
San Diego State University College of Engineering

ANFIS Responding Time

Inputs Membership-functions ANFIS Computation Time Response Time

1 2 3-4 16-18

3 7-8 20-23

4 9-11 23-26

5 13-15 26-29

6 17-19 30-32

7 22-23 36-38

2 2 12-14 26-28

3 30-31 44-46

4 80-82 94-96
San Diego State University College of Engineering

Conclusions and Future Work

• A MIMO ANFIS controller has been designed and tested through simulation
and experimental studies
• The desired controller can adaptively adjusting to system variations through
supervised and un-supervised learning.
• Future tasks include extending the MIMO design to a multiple inputs, two
output structure and evaluate the performance of this MIMO implementation
using Player/Stage and experimentation.
• We will add on-line learning functionality to our embedded MIMO ANFIS
that will effectively tune the parameters computed from off-line training data.
San Diego State University College of Engineering
San Diego State University College of Engineering