Robotics and Autonomous Systems 54 (2006) 1026–1038

www.elsevier.com/locate/robot

Automatic visual guidance of a forklift engaging a pallet

Michael Seelinger a,∗ , John-David Yoder b
a Yoder Software, Inc., 715 W. Michigan Ave., Urbana, IL 61801, United States
b Department of Mechanical Engineering, Ohio Northern University, Ada, OH 45810, United States

Received 4 June 2004; received in revised form 11 September 2005; accepted 2 October 2005
Available online 1 August 2006

Abstract

This paper presents the development of a prototype vision-guided forklift system for the automatic engagement of pallets. The system is
controlled using the visual guidance method of mobile camera-space manipulation, which is capable of achieving a high level of precision in
positioning and orienting mobile manipulator robots without relying on camera calibration. The paper contains development of the method, the
development of a prototype forklift as well as experimental results in actual pallet engagement tasks. The technology could be added to AGV
systems enabling them to engage arbitrarily located pallets. It also could be added to standard forklifts as an operator assist capability.
c 2006 Elsevier B.V. All rights reserved.

Keywords: Mobile manipulation; Visual guidance; Machine vision

1. Introduction
This paper presents the development of a prototype system
for the vision-guided automatic engagement of arbitrarily-
positioned standard pallets by a computer-controlled forklift.
The method presented here enables a robotic forklift vehicle
to engage pallets based on their actual current location by using
feedback from vision sensors that are part of the robotic forklift.
The development of this technology could advance the current
state of the art in material handling in two distinct ways.
First, the technology could be added to commercially avail-
able AGV (automatically guided vehicle) material handling
systems. Typically these systems use floor-embedded wires or
laser beacons located throughout their areas of operation to nav-
igate the AGVs through the warehouse and to align themselves
for pallet engagement. While most AGV systems are used for
horizontal transportation of material, leaving the task of storing
pallets in racks up to humanly driven standard forklifts, there
are some AGV systems capable of stowing and retrieving pal-
lets from rack locations. In order to engage a specific pallet,
the pallet’s vertical position (or rack height) as well as its po-
sition in the reference frame of the map must be known to the
∗ Corresponding author. Fax: +1 217 344 2987.
E-mail address: mseelinger@yodersoftware.com (M. Seelinger).
AGV system. AGVs are effective in engaging pallets that have
been placed in the racks by the AGVs themselves because the
AGVs are capable of placing the pallets to within 1 cm of the
desired location [1]. AGV pallet engagement assumes the pallet
is positioned to within 1 cm of the nominal desired location. If
a pallet is placed in the rack by a human operating a standard
forklift, there is no assurance that the pallet will be located pre-
cisely enough for an AGV system to engage the pallet. Other
factors can cause an AGV to lose the necessary precision for
pallet engagement such as disruption of a laser beacon signal
or loss of communication with the guiding wires or a degrada-
tion of the system’s calibration. (Note: background material on
AGV systems can be found in [1] and/or online at [2].)
The use of the vision-guided method for pallet engagement
presented here would provide AGV systems the flexibility to
perform their final positioning and engagement based on the
pallet’s actual location relative to the AGV. This would enable
human operators to place pallets amidst the previously AGV-
only racks, affording much more flexibility and interaction
between AGV material handling units and human operators on
standard forklifts. Also, the ability to engage pallets based on
their actual locations would enable AGV systems to operate
in less structured environments. For example, rather than only
working in carefully laid-out warehouses, AGVs could unload
pallets automatically off tractor-trailers, currently a very labor
intensive task.

c 2006 Elsevier B.V. All rights reserved.

0921-8890/$ - see front matter

doi:10.1016/j.robot.2005.10.009
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 1. Crown SC 3-wheeled electric forklift prior and after retrofit.
The second way that the vision-guided pallet engagement
method presented here could advance the current state of the
art would be to add it to standard, manually-guided forklifts.
The pallet engagement process can be difficult for forklift
drivers due to the fact that their line of sight is obstructed
by the mast of the forklift itself. This is especially true when
loading/unloading very high racks. The vision guidance control
method presented here could be added to standard forklifts to
automate the pallet-engagement portion of the forklift material
handling operation. The driver would have the responsibility of
navigating through the warehouse or other building, avoiding
obstacles and getting the forklift into view of the desired pallet.
Then, the operator could switch into ‘automatic-engagement
mode’ which would enable the system to engage the pallet
automatically. Such a system could reduce the amount of
product damage that occurs in forklift accidents involving pallet
engagement and transportation of product. Potentially, it could
also improve safety as it would allow the forklift to engage
pallets safely avoiding the possibility of knocking products off
pallets that are located very high off the ground.
The vision-guided method for forklift pallet engagement
presented in this paper is called ‘Mobile Camera-Space
Manipulation’ or simply ‘MCSM’. It was developed originally
for the visual guidance of a planetary exploration rover
equipped with a robotic arm [3] and has been adapted and
further developed to control the forklift prototype system
presented here and shown in Fig. 1. While the method requires
that at least two vision sensors be attached to the forklift
system, it does not require the system to maintain any sort of
strict calibration of the cameras relative to the system nor the
cameras relative to each other. Such calibrated relationships
would be difficult to maintain in the harsh climate typically
encountered by industrial forklifts. Likewise it would be
difficult to establish and maintain these calibrated relationships
on many forklift systems due to the low-precision (relative
to typical industrial robotics) nature of the mechanics of
these models. MCSM uses vision-based estimation along with
the nominal kinematic model of the forklift and a simple
camera model to achieve a high-level of positioning precision
in a robust fashion. This precision and robustness has been
1027
demonstrated by experimental results of the forklift prototype,
which will be presented in Section 6.
Following this introduction, the paper is divided into
five sections. Section 2 gives background for the reader to
understand the current state of the art in vision-guided pallet
engagement by forklift vehicles as well some other vision-
guided mobile manipulator systems. Section 3 gives some
necessary background information on standard CSM, which is
a basis for developing MCSM. Section 4 presents the prototype
forklift used in the development and testing of the method.
Section 5 presents how the method of MCSM is developed to
control the forklift. Such topics as target definition, trajectory
generation and tracking, and vision-tracking will be discussed.
The experimental results are given in Section 6. Section 7
contains the conclusions.
2. Background
MCSM is a method for controlling mobile manipulators
via vision sensors that are carried with the mobile system.
The term mobile manipulator refers to a system consisting
of a manipulator with holonomic degrees of freedom (DOFs)
mounted on a mobile base which has non-holonomic DOFs.
(See [4] for a description of the difference between holonomic
and non-holonomic constraints.) A forklift is an example of a
mobile manipulator. The forks of the forklift can be thought of
as a manipulator. Typical forklifts, including the one discussed
in this paper, have three holonomic fork DOFs: vertical,
sideshift, and tilt angle. The mobile base of the forklift has two
non-holonomic DOFs of control: the angle of the steering wheel
as well as the angle of the drive wheel(s). These can be used
to position the forklift in three DOFs: the (X, Y ) location of
the forklift in the plane parallel to ground as well as its angle
of orientation. Other examples of mobile manipulators include
exploration rovers equipped with robotic arms and construction
equipment such as backhoes or front loaders.
MCSM is an extension of standard Camera-Space Manipula-
tion (CSM), a control strategy that has proven through years of
testing to provide highly accurate control of holonomic systems
using fixed cameras [5–7]. Standard CSM has even been used
to control a mobile manipulator, a miniature ‘simulated’ fork-
lift [8,9]. However, standard CSM is ill-suited to control mobile
1028 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
manipulators since it is limited by the need to control the mo-
bile system using stationary cameras, which severely limits the
workspace of the mobile system. MCSM overcomes this limita-
tion of standard CSM since it allows the cameras to move with
the mobile manipulator.
Pagès et al. present a computer vision system for automatic
forklift pallet engagement [10]. Their method incorporates the
use of a single calibrated camera. Their work focuses more
on the pallet recognition method of the vision system rather
than on the overall visual control of the forklift system itself.
Currently, their system is configured to engage pallets that are
located on the ground level only. The trajectory generation
method presented by Pagès is primitive. It incorporates a
scheme of turn, drive straight, turn, drive straight. While this
method of trajectory generation does allow the forklift to
engage the pallet, it is slow and impractical as the system must
frequently stop and start. Pagès presents some experimental
results, however as with Miller [9], the vehicle used was not
an actual forklift but a small mobile robot with forks attached
to the manipulator intended to simulate the motion of an actual
forklift. The system presented in this paper differs in several
respects. MCSM requires the use of at least two cameras;
however, it does not require camera calibration. The strategy for
pallet recognition employed with MCSM differs substantially
from Pagès’ system. Also, the MCSM system is not limited to
engaging pallets that are located only on the floor. Unlike the
system of Pagés et al. the trajectory generation and tracking
method used with MCSM enables our system to engage a pallet
much the same way a human operator would drive a forklift
without the need for stopping and starting.
The only other vision-guided forklift system that the authors
are aware of is the Automated Material Transport System which
is under development at the National Robotics Engineering
Consortium [11]. The development of this system seems
to rely heavily on computer vision both for the navigation
throughout the warehouse as well as the engagement of pallets.
Unfortunately, there is no public information available that
describes the methods employed for pallet engagement.
Several manufacturers provide forklift-style AGV systems
[12–15]. Product information available through company
websites focus on load capacity, fork reach, repeatability, and
in the case of warehousing applications the aisle width in which
the AGV can operate. Currently AGV companies do not offer
vision systems as part of their control package for the automatic
engagement of as-located pallets.
Other examples of vision-guided mobile manipulators in
the literature include systems proposed by Pissard-Gibollet
and Rives, and Swain and Devy [16,17]. The goal of their
system is to navigate the mobile robot using a vision sensor
held by a manipulator mounted on the mobile base. They
do not address the task of actually engaging an object with
the manipulator mounted on the mobile platform. MacKenzie
and Arkin describe a vision-guided mobile manipulator for
drum sampling [18]. This system servos the mobile base into
position using a camera mounted on the manipulator’s waist.
Then the manipulator engages a drum via a visual servoing
technique guided by a separate vision sensor with an eye-
in-hand configuration. MCSM differs from MacKenzie and
Arkin’s approach in several respects. For instance, MCSM uses
the same vision sensors for navigating the mobile platform
as well as positioning of the manipulator. Also, MCSM’s
visual control approach is based on an open-loop method
rather than the closed-loop visual servoing technique. The
advantages of the open-loop approach over closed-loop visual
servoing techniques are discussed in [19]. Tsakiris, Rives,
and Samson present a vision-guided mobile manipulator also
based on visual servoing techniques [20]. An eye-in-hand
vision system configuration guides the mobile robot and its
onboard multiple DOF manipulator. The goal of their research
is to present methods for achieving stable pose control of
both the non-holonomic base as well the manipulator mounted
camera, which they achieve by the addition of holonomic
DOFs for positioning the camera. Aside from the differences
in open loop versus closed loop control, their system uses only
one camera and is mounted in the eye-in-hand configuration
whereas MCSM systems use at least two cameras neither of
which is mounted on the manipulator.
3. MCSM as an extension of standard CSM
Standard CSM plays an integral role as a subsystem of the
MCSM forklift system. While a more complete description of
standard CSM is given in [5,6], it is important to discuss certain
basic CSM principles in order to understand the MCSM forklift
system. The first part of this section develops CSM as it is used
to control purely holonomic systems. The second part of this
section describes how standard CSM has been used to control a
‘simulated’ forklift mobile manipulator.
3.1. Standard holonomic CSM
In typical implementations of standard CSM for controlling
holonomic manipulators CSM uses widely separated cameras
located remotely from the robot they control. CSM works by
estimating a relationship in the reference frame of each camera
between the appearance of image-plane visual features located
on the manipulator with the internal joint configuration of the
robot. This relationship, f , is based on the orthographic camera
model [21] and is described with a set of view parameters given
by C = [C1 , C2 , . . . , C6 ]T :
xc = f x (C, 2) = (C12 + C22 + C32 + C42 )X (2)
+ 2(C2 C3 + C1 C4 )Y (2)
+ 2(C2 C4 − C1 C3 )Z (2) + C5
(1)
yc = f y (C, 2) = 2(C2 C3 − C1 C4 )X (2)
+ (C12 − C22 + C32 − C42 )Y (2)
+ 2(C2 C4 + C1 C3 )Z (2) + C6
where (xc , yc ) is the position in the reference frame of one
of the cameras and 2 is the pose of the manipulator (see
Fig. 2 for definition of coordinate systems). The position of
a point on the manipulator measured relative to the (X, Y, Z )
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 2. Coordinate system definitions for robot and cameras.
coordinate system is fixed to the base of the manipulator
as can be seen in Fig. 2. It is dependent upon the nominal
kinematic model of the robot and its corresponding pose, 2.
The view parameters are estimated based on sample pairs of
robot internal joint configuration and camera-space location of
manipulator features. The minimization of
ns h
X
2
2 i
Γ = xck − f x (C, 2) + yck − f y (C, 2) Wk
k=1
over all C provides the necessary conditions to solve for the six
view parameters for each camera. In Eq. (2) n s denotes the total
number of joint-space/camera-space samples. The Wk of Eq. (2)
is a relative weighting factor that enables the system to place
a higher relative weight on samples acquired near a particular
region of interest. When all samples are weighted equally, for
instance in the initialization of the view parameters, Wk = 1.
When the manipulator approaches its target incoming samples
might be assigned a weight of Wk = 10 or Wk = 20 or some
other weight higher than samples acquired more distant from
the current pose. This skews the view parameter relationship
towards the more heavily weighted data so that the system
is more accurate in this heavily weighted region of joint and
camera space.
While CSM uses the orthographic camera model and the
nominal kinematic model of the robot, it still can be stated
truthfully that CSM, and therefore MCSM as well, is a
calibration-free method. The reasons for this derive from the
use of estimation and the fact that the system automatically
skews the relationships described by Eq. (1) such that they work
well in a local region of operation. In other words, Eq. (1) could
describe a calibrated relationship if the view parameters were
established and held fixed. However, these view parameters
change frequently either through updating with new visual
information or simply by skewing the information used to
estimate them such that the relationship described by Eq. (1)
1029
holds in a localized region of joint-space and camera-space
operation. Due to the fact that the measurement and positioning
are carried out in the same reference frames (those of the
participating cameras) CSM compensates for the inadequacies
of the orthographic camera model as well as inaccuracies in
the kinematic model of the robot. Even if the kinematics of the
system were to be altered during a given operation (for instance
if a joint is deflected by the load it carries) CSM can compensate
by simply acquiring more joint-space/camera-space samples.
Experimental results demonstrating this ability are presented
in [3].
Additionally, a process called ‘flattening’ has been
developed to increase the precision of CSM/MCSM systems
by correcting for the inadequacies of the orthographic camera
model. Flattening uses orthographic camera model parameters
and a rough estimate of the distance between the robot reference
frame, (X, Y, Z ) in Fig. 2, and camera reference frames,
(X c1 , Yc1 , Z c1 ) and (X c2 , Yc2 , Z c2 ) in Fig. 2, to obtain identical
results as would have been achieved by using the more accurate
‘pin-hole’ camera model (see [7] for more detail).
3.2. Non-holonomic CSM
A form of standard CSM called ‘non-holonomic CSM’ has
been used to control a small mobile manipulator, a ‘simulated
forklift’, that is capable of engaging a small pallet [8,9]. MCSM
was developed, in part, to overcome some of the restrictions
and disadvantages of using this form of CSM to control mobile
manipulators. CSM requires a set of cameras that are stationary
(2) relative to the mobile manipulator system. This severely limits
the workspace of a mobile system and makes the use of
CSM impractical for the task of pallet engagement by an
industrial forklift. CSM’s need for stationary cameras creates
other complications such as reduced camera resolution as the
system moves further from the cameras, the possibility of
the system itself obstructing the view of the camera(s), and
maintaining communication between the remote cameras and
the mobile vehicle. The relationship between the physical,
3D position of manipulator features and their corresponding
camera-space locations as described in Eq. (1) now depends
on both the holonomic DOFs of the manipulator and the non-
holonomic DOFs of the mobile base.
3.3. Introduction to MCSM
MCSM retains the strengths of CSM while removing the
restrictions and disadvantages of using CSM to control mobile
manipulators. MCSM allows the cameras to be mounted to
the mobile platform. With MCSM the relationship between
the physical, 3D position of manipulator features and their
corresponding camera-space locations as described in Eq. (1) is
purely holonomic, whereas with standard CSM the relationship
depends on both the holonomic and non-holonomic DOFs.
There are several advantages to this. First, the view parameters
can be found without numerical integration of differential
relationships. This simplifies the view parameter estimation
procedure and the trajectory updating procedure in addition
1030 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
to reducing the computational cost. The separation of the
non-holonomic information from the holonomic information
in the estimation model increases the accuracy of the view
parameter estimates. Due to problems such as wheel slip
and numerical estimation errors, information from the non-
holonomic DOFs tends to be much less accurate and less
reliable than information from the holonomic DOFs. With
standard CSM, the accuracy of the view parameter estimates
and therefore of the overall precision of the system is dependent
upon the relatively inaccurate non-holonomic information. In
contrast the overall precision of MCSM systems is not affected
by any non-holonomic inaccuracies.
In MCSM systems the camera locations are not known
precisely relative to the rest of the system nor to each other.
The system performs equally well over a large range of
camera locations. The position and orientation has changed
substantially as well. As visual data is acquired, the system
adapts to the current location of each camera. In contrast,
calibrated vision systems for robot guidance require that the
position and orientation of the cameras be known relative to
each other as well as relative to the robot (see for instance
[22,23] for examples of calibrated vision systems). As the
calibration of the system degrades the positioning precision
of the system degrades substantially. External factors such as
vibration, thermal gradients, or physical damage to the system
can easily cause a system to fall out of calibration. It would be
difficult if not impossible to maintain calibration if a calibrated
vision system were to be used to control a typical industrial
forklift.
4. Forklift prototype
This section gives a brief description of the forklift prototype
used for developing and testing MCSM. A used Crown SC
series three-wheeled electric lift truck with load capacity of
3200 lbs and vertical reach of 24 ft was acquired for use as
the prototype. In order to use MCSM to control the forklift,
cameras had to be added to the system and the forklift had to
be modified to enable computer control of all of its actuators.
Fig. 1 shows the forklift prior to and after retrofit.
A platform for housing the computer and controls was
built on top of the forklift’s cage. The computer that runs the
system has a 450 MHz Celeron processor and runs Windows
NT 4.0. An Acroloop 8000 8-axis motion control card with
break-out box controls the motion of all of the actuators,
reads all of the analog feedback devices, and controls several
input/output channels. The Sony XC-75 cameras with wide-
angle lenses are controlled using two Imagenation PX-610
frame grabbers each having resolution of 640×480 pixels. Five
IDC (Industrial Devices Corporation) linear actuators are used
to control various functions of the forklift. Three of the five are
used to move the forklift’s hydraulic valves that control motion
of the forks in the vertical, sideshift, and tilt angle directions.
One linear actuator is used to engage the accelerator, the other
to engage the brake. These five linear actuators are powered
with DC power supplies and Copley amplifiers. A Kollmorgen
motor is used to control the angle of the steering wheel. In
addition to these actuators, there are three analog feedback
devices that are used by the system. Two of these devices are
Penny + Giles Controls Inc. linear potentiometers, which give
feedback regarding the sideshift position of the forks as well
as the current tilt angle of the forks. The third analog feedback
device is a UniMeasure, Inc. linear position transducer. This
device gives the system feedback regarding the vertical position
of the forks. Optical encoders are mounted via rolling contact
on each of the two drive wheels. These give the system feedback
on exactly how the drive wheels have moved. Assuming no slip,
this information can be used in a numerical integration scheme
to determine the path that the forklift has actually traversed.
Computer control of the forklift is divided into three
categories: control of the fork position, control of the forklift
speed, and control of the forklift steering. Motion control
programs incorporating PID control laws were written for
the Acroloop controller in its own language, AcroBasic.
Through experimentation it was determined that the computer
system is able to control the position of the forks to within
1
3 in. This precision is a function of this particular forklift
prototype system and not a limitation of MCSM or of MCSM
implemented on forklifts in general. (MCSM was used in [3]
to achieve seven times more precise positioning of a mobile
manipulator than the 13 in. reported here.)
Also, through experimentation it was determined that there
was a significant amount of ‘play’ in the steering system.
While a nominal relationship exists between the position of the
steering motor and the position of the steering wheel (there is no
direct feedback of the wheel angle), it was determined that there
was up to 10◦ of play between the nominal target position for
the steering angle and the effective steering angle. It is possible
to use information from the optical encoders mounted to the
drive wheels to find the effective steering angle after a motion
has begun. Using this information, a correction routine was
added to the Acroloop steering program in an effort to improve
the accuracy of the steering of the system. As the forklift
moves, it monitors the effective steering angle, compares it to
the desired steering angle, and issues corrections to the steering
motor. This procedure is described in more detail in Section 5.5.
It is recognized that this prototype would be inadequate
for actual industrial use. For instance, the wiring is exposed,
which would be very dangerous if operated under the normal
industrial operating conditions. Also, the system is currently
limited in speed. This is due primarily to the inadequacy of the
hardware chosen for the prototype. While inadequate for actual
industrial use in its current state, the forklift prototype has
served its purpose in demonstrating the efficacy of the vision-
guided method presented in this paper.
Schematic views of the forklift prototype are given in Fig. 3.
The three holonomic DOFs of the forks are characterized by θv
for the vertical direction, θs for the sideshift motion, and θt for
the angle of tilt. The angular position of the two drive wheels are
θ1 and θ2 respectively. The steering angle of the rear wheel of
the forklift is represented as γ . The angles, θt , θ1 , θ2 , and γ are
measured in radians and the quantities θv and θs are measured in
inches. The distance from the rear steering wheel and the front
axle is denoted as da where da = 52.25 in. The half length
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 3. Top and side views of forklift prototype schematic.
between the two drive wheels is denoted as db where db =
17.45 in. The distance from the front axle to the end of the forks
is denoted as d f where d f = 56.0 in. The radius of the drive
wheels is denoted as R where R = 6.66 in. It is important to
note that the (X t , Yt , Z t ) coordinate system, as shown in Fig. 3,
is located midway between the forks and travels with the forks.
The (X w , Yw , Z w ) coordinate system is fixed midway between
the two front wheels. It is possible to transform any point
from the (X t , Yt , Z t ) coordinate system to the (X w , Yw , Z w )
coordinate system using the forward kinematics:
X w (2) = (cos θt − 0.01 sin θt )X t − (0.01 cos θt + sin θt )Z t
+ 59.41 cos θt + (6.28 − θv ) sin θt − 3.41
Yw (2) = Yt + θs (3)
Z w (2) = (sin θt + 0.01 cos θt )X t + (cos θt − 0.01 sin θt )Z t
+ 59.41 sin θt + (θv − 6.28) cos θt + 0.8
where 2 = (θv , θs , θt ). The functions X w (2), Yw (2), and
Z w (2) correspond to the position vector functions X (2),
1031
Y (2), and Z (2) used in Eq. (1). It should be noted that
several parameters describing the physical layout of the system
have been combined into numerical constants in Eq. (3) for
simplicity.
5. MCSM implementation on forklift prototype
The task for which MCSM was implemented on the
forklift prototype was a simple engagement of an as-located
pallet. The strategy for engaging the pallet with the MCSM
algorithm is broken down as follows. The first task is the
visual identification of the pallet. Two methods for visual
identification are discussed. Once the pallet is identified
camera-space targets are generated. These targets are used to
resolve the three fork DOFs necessary to engage the pallet.
Also, the targets are used to create a trajectory for the forklift
to follow. As the forklift moves towards the target, additional
visual samples are acquired. The targets, and subsequently,
the target fork pose and trajectory are updated with new
information as it becomes available from the acquisition and
1032 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 4. Pallet with fiducials placed on it.
processing of new images. Also, the system creates a new
nominal trajectory when it detects that it has failed to track
the ideal trajectory with sufficient accuracy to ensure pallet
engagement.
5.1. Pallet identification
The most effective and robust method of identification of
the pallet is to place features (also called fiducials) on the
pallets themselves that simplify the visual identification of
the pallet. The fiducials used for this project take the shape
Fig. 5. Photo of pallet and edge detection run on pallet photo.
Fig. 6. Image sequence for identifying lower pallet edge.
of concentric black and white circles. As shown in Fig. 4
three such fiducials are placed on each pallet: a white-centered
fiducial in the middle and two black-centered fiducials on either
side of the pallet. The forklift system is able to identify the
pallet if it can locate the white-centered fiducial and at least
one of the black-centered fiducials in both of its cameras.
While it is a bit of a concession to have fiducials on pallets as
opposed to using the pallet’s natural features, it should be noted
that this is not necessarily an unreasonable nor prohibitively
expensive proposition. Many warehousing operations purchase
custom made pallets to meet their particular needs. It would be
relatively simple to make fiducials part of custom-made pallets.
YSI has discussed this possibility with a pallet manufacturer
who indicated that adding fiducials similar to those employed
in our experiments to custom pallets would be quite simple and
would not drive up the cost of the pallets significantly.
The authors developed a simple method for pallet recog-
nition using only natural features for the purpose of demon-
strating that MCSM is not limited by a need for fiducials
being placed on the pallets. (Another natural feature recogni-
tion method was proposed by Pagès et al. [10].) Our method
uses a standard library for edge detection [24]. Fig. 5 shows a
picture of the pallet itself and next to it the edges that the edge
detection routine found. The system first looks for the edge seg-
ments that are of a certain minimum length. An example of this
can be seen in the left image of Fig. 6 where the white lines in-
dicate edge segments found. Through a series of matching tests
it tries to combine segments that are actually part of the same
continuous line (see middle image of Fig. 6). Finally, the system
identifies the lower edge of the pallet (see right image of Fig. 6).
This is done based on four conditions: (1) the minimum length
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038 1033

over all (X wm , Ywm , Z wm ). The variable n c represents the

number of cameras, which in the case of the current forklift
system is two. It should be noted that MCSM is not limited to
two cameras for operation; it can incorporate information from
as many cameras as are present in the system. Similar to Wk
of Eq. (2) W j is a relative weighting factor. In practice, W j is
always set to 1. However it could be varied if more than two
Fig. 7. Sketch of pallet with key points located via natural features detection. cameras were present or if it was known that information from
one or the other camera was not as accurate and thus should be
of the line itself, and the presence of another upper edge of (2)
de-weighted.
approximately the same slope and (3) same length (4) located
For the fiducial-assisted pallet identification process the
directly above the lower pallet edge. Fig. 7 shows a sketch of the
pallet vector is found as follows. Consider the case that both the
lower pallet edge which begins at corner 1 and runs to corner 2.
white-centered fiducial and the black-centered fiducial located
While this algorithm is simplistic, it did serve the purpose
to its left are identified in the pallet identification stage. The
of demonstrating MCSM’s ability to identify the pallet without
location of the white-centered fiducial is the same as the pallet
relying on artificial fiducials. In order to use the MCSM method
mid-point, (X wm , Ywm , Z wm ). The location of the left, black-
without fiducials in an industrial setting this natural features
centered fiducial is found using the same process as the pallet
pallet recognition routine will have to be made more robust.
mid-point, and is represented as (X wbl , Ywbl , Z wbl ). The pallet
5.2. Target definition vector, P̂v is then found by:
q
Once the pallet has been identified in the images taken by d P̂v = (X wbl − X wm )2 + (Ywbl − Ywm )2 + (Z wbl − Z wm )2
the forklift’s two cameras the next step is to create targets for X wbl − X wm Yw − Ywm Z wbl − Z wm (5)
engagement. Fig. 8 shows a schematic of the forklift and a P̂V = î + bl ĵ + k̂.
d P̂v d P̂v d P̂v
target trajectory for engaging the pallet. There are two pieces
of information about the pallet that are necessary for trajectory For the case of the natural features pallet identification
generation: the pallet mid-point and the pallet vector (both can process the pallet mid-point is found as follows. Using a
be seen on the right side of Fig. 8). Regardless of the method similar form of Eq. (4) it is possible to find the location
employed for pallet identification, it is necessary to find the of the two corner points in the (X w , Yw , Z w ) reference
location of the pallet mid-point and the pallet vector in the frame: (X wcp1 , Ywcp1 , Z wcp1 ) and (X wcp2 , Ywcp2 , Z wcp2 ). Then,
(X w , Yw , Z w ) coordinate system. the pallet mid-point, (X wm , Ywm , Z wm ) is found by:
At this point in the process it is assumed that the system
X wcp1 + X wcp2
has already established view parameters for each camera using X wm =
information obtained from joint-space/camera-space sample 2
pairs (see Section 3 for details). The pallet mid-point has Ywcp1 + Ywcp2
Ywm = (6)
been identified in both camera spaces and is represented as 2
(xc1m , yc1m ) for the first camera and (xc2m , yc2m ) for the second. Z wcp1 + Z wcp2
Z wm = + 3.0.
Using the view parameters and the relationships described in 2
Eq. (1) it is possible to estimate the pallet mid-point location It should be noted that Z component of the pallet midpoint is
in the (X w , Yw , Z w ) reference frame. This is accomplished by located approximately 3 in. above the lower pallet edge.
minimizing The pallet vector is found in a similar fashion as before:
n c  2
j
X q
Γ2 = xcm − f x (C j , X wm , Ywm , Z wm ) d P̂ =
v
(X wcp1 − X wcp2 )2 + (Ywcp1 − Ywcp2 )2 + (Z wcp1 − Z wcp2 )2
j=1 X wcp1 − X wcp2 Ywcp1 − Ywcp2 Z wcp1 − Z wcp2 (7)
2  P̂v = î + ĵ + k̂.

j d P̂ d P̂ d P̂
+ ycm − f y (C j , X wm , Ywm , Z wm ) Wj (4) v v v

Fig. 8. Forklift trajectory for pallet engagement.

1034 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 9. Error in tracking the ideal polynomial trajectory.
5.3. Resolution of fork DOFs
This subsection will review the steps necessary for
determining a suitable pose for the three fork DOFs that will
enable the forklift to engage the pallet. First it is assumed that
the pallet is positioned on the plane parallel to the ground. This
is a reasonable assumption as most warehouse floors as well
as pallet racks are relatively level. Thus, the tilt angle of the
forks is set to θt = 0. Next the vertical position of the forks is
resolved by simply necessitating that the forks be aligned with
the pallet mid-point. Thus, θv = Z wm . The sideshift position of
the forks must also be resolved. At the beginning of a trajectory,
this position is held to θs = 0. In theory, the steering and
drive wheels should suffice to line the forks up with the proper
orientation. However, once the forklift gets very close to the
engagement of the pallet, it is possible to use the sideshift DOF
of the forks to correct for any misalignment that might have
occurred in the trajectory tracking portion of the process. In this
case, the sideshift position of the forks is found to be θs = Ywm .
It should be noted that the sideshift axis of the forks can move
through a range of about seven inches.
5.4. Trajectory generation
The method for trajectory generation is similar to the
approach developed in [25]. The system makes use of a fifth
order polynomial to generate a suitable trajectory for the forklift
to follow that will position and align the forklift properly for
pallet engagement:
2
Yw = a0 + a1 X w + a2 X w 3
+ a3 X w 4
+ a4 X w 5
+ a5 X w .
The angles, φ and γ (as shown in Fig. 8), are related to the
polynomial by:
d2 yw
dyw da dxw 2
tan φ = , tan γ = − 2 .
dxw
 through experimentation that as the forklift gets closer to the
1 + dy w
dxw
The following six conditions are used to solve for
the parameters of the polynomial function. The first three
conditions are simply the initial conditions that when X w = 0:
Yw = 0, φ = 0, γ = γi . (10)
The initial steering angle, γi , is not necessarily equal to zero.
This is due to the fact that new trajectories are planned as
the forklift moves towards the pallet and acquires more visual
information. At the beginning of the trajectory it is likely
that the steering angle is not zero, but some known quantity
labeled, γi . The final position and orientation of the forklift
necessary to engage the pallet satisfy two more conditions.
The fourth condition is based on the final position that when
X w = X wm − d f :
Yw = Ywm (11)
where d f is shown in Fig. 3. The fifth condition is the final
orientation. It is written as when X w = X wm − d f :
Ywbl − Ywm
φ = φ f = ds cos−1 . (12)
d P̂v
The parameter, ds , is simply used to get the correct sign for φ f .
X wbl −X wm
If d P̂ > 0 then ds = −1, otherwise ds = 1. Note that
v
when the forklift is in the final position the Ya direction shown
in Fig. 9 is perfectly aligned with the pallet vector shown in
Fig. 8.
The sixth condition is met by stipulating that the forklift
must make the final approach to the pallet in almost a straight
line. Fig. 8 shows a trajectory for the forklift in which the last
segment of the trajectory is an almost straight-line approach.
This reduces the amount of turning that the forklift must carry
out at the end of the trajectory, which increases the system’s
engagement precision. This condition is written as:
(8) d2 Yw X wm − d f


= 0. (13)
dxw2
Due to the restrictive conditions associated with the fifth
order polynomial planner the forklift sometimes has to undergo
sharp steering changes in small spaces. It has been determined
(9)
pallet, it is possible to use a third order polynomial planner
instead of the fifth order one. The lower order polynomial
planner usually reduces the amount of abrupt and large changes
in the steering angle. The two restrictions that are removed in
the case of the third order polynomial planner are conditions
four and six which refer to Eqs. (11) and (13) respectively. Due
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
to the removal of condition four the forklift may not end up
aligned perfectly in the Yw direction, though the orientation is
correct. In this case, the sideshift DOF of the fork may be used
to correct this misalignment.
5.5. Visual updating and trajectory correction
As the forklift moves towards the pallet, the system will
update its trajectory several times. These updates may be due to
incoming visual information or by developing a path-tracking
error over the specified tolerance. The equations of motion of
the forklift along with samples from the drive wheel encoders
are used to keep track of the actual motion that the forklift
has undergone. This is carried out as follows, based on the
development in [26].
First, the parameters, α and u, are calculated using
knowledge of the angular position of the two front wheels, θ1
and θ2 :
θ1 + θ2 1θ2 − 1θ1
α= , u= . (14)
2 1θ2 + 1θ1
The equations of motion, based on the non-holonomic
constraints, can be written as:
dX a dYa dφa uR
= R cos φ, = R sin φ, = (15)
dα dα dα db
where the angle of orientation of the forklift, φ, is shown in
Fig. 8 and the wheel radius, R, and the parameter, db , are shown
in Fig. 3. The actual position and orientation, (X a , Ya , φa ) are
measured relative to the (X w , Yw , Z w ) coordinate system with
the caveat that until the trajectory is updated the origin of the
(X w , Yw , Z w ) remains fixed even though the actual forklift
system is in motion.
As the forklift moves forward a small distance, the current
position of the forklift is approximated using a simple
numerical integration of these equations of motion:
X ai+1 = X ai + 1α R cos φai
(16)
Yai+1 = Yai + 1α R sin φai
1αu R
φai+1 = φai + (17)
db
where 1α = 1θ1 +1θ 2
2
. trajectory.
Using the actual position and orientation of the forklift
it is possible to determine an error in the tracking of the
ideal polynomial trajectory. There are two error components
used: the error normal to the polynomial, E n , and the error in
orientation, E φ . Fig. 9 shows the front axle of the forklift in
reference to the ideal polynomial trajectory. The actual position
based on Eqs. (16) and (17) is denoted as (X a , Ya , φa ). The
X -component of the point at which a line drawn along the
axle intersects the ideal polynomial is denoted as, X p . This
intersection point can be approximated with a simple iterative
procedure requiring only three iterations. This geometric basis
for computing error is used rather than a more typical time-
based strategy for the reasons discussed in [27]. Once X p is
1035
known the value of Y p is found with Eq. (8). Then, the error
normal to the path is calculated as:
q
E n = (X a − X p )2 + (Ya − Y p )2 . (18)
The ideal orientation of the forklift at point, (X p , Y p ), is
denoted, φ p , and is computed using Eq. (9). The error in
orientation is simply:
E φ = φa − φ p . (19)
There are three indicators that will cause the system to
update its trajectory. The first is simply distance traveled. If
the actual distance traveled, X a , grows beyond a set amount
(typically 20 in.), the system will begin a trajectory update.
If the error normal to the polynomial, E n , increases above its
tolerance (typically 3 in.) then the system will also update. If
the orientation error, E φ , grows beyond its tolerance (typically
5◦ ) the system will update its trajectory.
To create an updated trajectory plan, the system first attempts
to identify the pallet using its vision sensors. If it can,
the system updates the trajectory based on this new pallet
information. In some instances in order to have the forklift
approach the pallet with the proper orientation, the original
trajectory calls for the forklift to turn away from the pallet at
the outset of the trajectory. During periods when the forklift is
turned away from the pallet it is not possible to perform a visual
update. When this occurs the system uses the history of actual
forklift motion to create a new trajectory plan, which corrects
for the errors incurred.
The forklift continues updating the trajectory in this manner
until it reaches the final position with the proper alignment.
Then, the forklift moves straight ahead a fixed distance based
on the length of its forks to engage the pallet.
It should be noted that there are two other factors that are
used to increase the precision of the system. The first factor
is inherent to the method of MCSM. When the forks move
towards the final pose, the pose necessary to engage the pallet,
additional video samples may become available. These samples
can be used with Eq. (2) to update the view parameters, skewing
the relationship of Eq. (1) to be more accurate in the local region
of joint and camera space. With updated view parameters,
the system updates its estimates of the location of the pallet
mid-point and the pallet vector. With the updated information
about the pallet it is sometimes necessary to update the target
The second factor used to increase the precision of the
system has to do with the steering angle of this particular
forklift system, γ . Through experimentation it has been
determined that there is approximately ten degrees of ‘play’
in the steering angle. With this large amount of ‘play’, it
is possible that the commanded steering angle may differ
substantially from the effective steering angle. This discrepancy
will cause E n and E φ to grow quite quickly. With information
from the actual history of wheel rotations, θ1 and θ2 it is
possible to estimate the current effective steering angle by:
da θ̇1 − θ̇2


tan γ =
. (20)
db θ̇1 + θ̇2
1036 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
Fig. 10. Forklift engaging a pallet.
If this quantity differs substantially from the currently
commanded steering angle, then the system issues a small
correction in an attempt to close the difference between
the effective and commanded steering angles. It should be
emphasized that this discrepancy is not an inherent quality of
MCSM, but a mechanical limitation of the current prototype
forklift used for testing.
6. Experimental results
Two types of tests were run for pallet engagement: one
with the fiducials on the pallet and the other using only the
natural features of the pallet. Fig. 10 shows several pictures
of the forklift performing a typical pallet engaging experiment
with the fiducials on the pallet. For the purposes of the pallet
engagement task, success was measured by achieving pallet
engagement. For a more detailed analysis of the precision of
MCSM systems see [3].
For the case of using the fiducials on the pallets a series
of 100 test runs were carried out. For these tests the initial
forklift position required that the pallet be in view of both of
the forklift cameras. The initial location of the pallet relative
to the forklift varied from roughly 6 ft to 12 ft in the X w
direction (see Fig. 3), −3 ft to 3 ft in the Yw direction, and 0
to 4 ft in the Z w direction. The initial angle of the pallet in
the X w − Yw plane relative to the forklift varied from roughly
−20◦ to 20◦ . The forklift successfully engaged the pallet in
98 of the 100 trials. In the two unsuccessful tests, the system
automatically detected that it could not find the pallet and
stopped itself waiting for assistance. In roughly 20% of the tests
the forklift was unable to engage the pallet on its first attempt.
Once the system determined that it was too close to the pallet in
order to correct its final position and/or orientation it backed up
automatically to give itself sufficient space to carry out a new
trajectory. The primary source of error which causes the system
to have to back up is the ‘play’ in the steering angle.
A series of tests were carried out using the natural features
pallet identification routine instead of the fiducials on the pallet
to demonstrate that MCSM is not limited to using artificial
features for visual identification and engagement of pallets.
However, it should be noted that in order for the natural
features routine to identify the desired pallet successfully the
environment surrounding the pallet was cleared of other pallets
or objects that might have ‘confused’ the algorithm. In these
tests the system successfully engaged the pallet in 11 of 13
trials. While the natural features algorithm will have to be made
much more robust to be used in an actual industrial setting,
these tests demonstrate that the system is capable of engaging
pallets using only their natural features.
7. Conclusion
We have presented the development of a prototype vision-
guided forklift system for the automatic engagement of pallets.
The system is controlled using the visual guidance method
of mobile camera-space manipulation (MCSM), which was
developed originally for use with planetary exploration rovers.
MCSM is capable of achieving a high level of precision in
positioning and orienting mobile manipulator robots such as
rovers or in the case of this paper a modified, computer-
controlled forklift. It achieves this precision without relying
on camera calibration. MCSM is a significant advancement
over standard camera-space manipulation (CSM). It enables
the mobile system to carry its cameras onboard, whereas CSM
required the cameras to be stationary. Aside from removing
the limitation in workspace area caused by the requirement of
stationary cameras – a limitation that makes CSM impractical
for the control of industrial forklift systems – the ability of
 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
MCSM systems to carry the cameras onboard solves other
potential problems such as reduced camera resolution, visual
obstruction of the target by the system itself, and the need
to maintain reliable and robust communication between the
cameras and the mobile system. Also, MCSM’s estimation
model is not adversely effected by the relatively inaccurate
information from the non-holonomic DOFs as is the case with
CSM systems.
The development of this technology for forklift-type systems
could advance the current state of the art in material handling
in two distinct ways. First, the technology could be added to
commercially available AGV material handling systems. This
would enable AGV systems to engage pallets based on their
actual location as opposed to the current practice of engaging
pallets on the assumption that they are located within 1 cm of
the previously recorded position. This would also give AGV
systems the capability of performing tasks such as tractor-
trailer unloading where it is impossible to know the position
of the pallets a priori. The technology developed here could
also be added to standard industrial forklifts creating a ‘semi-
autonomous’ forklift. A forklift operator would still have the
task of navigating the forklift through the warehouse and
bringing the pallet in view of the forklift’s cameras. Then,
MCSM would perform the final positioning and engagement of
the pallet. This technology potentially could reduce the amount
of product damage that occurs in forklift accidents involving
pallet engagement and transportation of product.
This paper describes an actual prototype forklift system
and two types of pallet engagement experiments. With
the assistance of fiducials placed on pallets for visual
recognition, the prototype system was capable of automatic
pallet engagement in 98% of the experiments conducted.
A rough algorithm was developed to enable the system to
automatically detect and engage pallets without the need for
artificial features.
The technology presented in this paper will be developed
further in several directions. First, in an effort to commercialize
the technology the authors plan to continue their work in
making the system more robust by increasing the reliability
as close to 100% as possible. The authors also hope to adapt
the method and system for high reach applications. Work is
currently being undertaken to develop a framework for the
cooperation of multiple mobile manipulators, at least one of
which will be controlled using the MCSM method. The initial
development of the cooperative control includes testing the
forklift system presented in this paper cooperating with a small,
high-precision rover.
Acknowledgments
The authors would like to thank NASA-JPL for support
of the development of this technology through SBIR Phase I
contract NAS8-98170 and Phase II contract NAS3-99131,
particularly our technical monitor, Eric Baumgartner, and our
SBIR coordinator, Patricia MacGuire. The authors also thank
Steven Skaar from the University of Notre Dame, Bob Emery,
Hal Ulrich, Loren Shaum of Automation Solutions, Inc., and
1037
Sanjiv Singh of Carnegie Mellon for their assistance in the
successful completion of the Phase II contract.
References
[1] G.A. Castleberry, The AGV Handbook, Braun-Brumfield, Inc., 1991.
[2] MHIA Elessons on Automatic Guided Vehicle Systems,
http://www.mhia.org/psc/PSC Products GuidedVehicle elessons.cfm.
[3] M. Seelinger, J.-D. Yoder, E.T. Baumgartner, S.B. Skaar, High-precision
visual control of mobile manipulators, IEEE Transactions on Robotics and
Automation 18 (6) (2002) 957–965.
[4] H. Seraji, Configuration control of rover-mounted manipulators, in:
Proceedings of IEEE Intl. Conf. on Robotics and Automation, 1995, pp.
2261–2266.
[5] S.B. Skaar, W.H. Brockman, R. Hanson, Camera space manipulation,
International Journal of Robotics Research 6 (4) (1987) 20–32.
[6] S.B. Skaar, W.H. Brockman, W.S. Jang, Three dimensional camera space
manipulation, International Journal of Robotics Research 9 (4) (1990)
22–39.
[7] E.J. Gonzalez-Galvan, S.B. Skaar, U.A. Korde, W.Z. Chen, Application
of a precision enhancing measure in 3-d rigid-body positioning using
camera-space manipulation, International Journal of Robotics Research
16 (2) (1997) 240–257.
[8] S.B. Skaar, I. Yalda-Mooshabad, W.H. Brockman, Nonholonomic
camera-space manipulation, IEEE Transactions on Robotics and
Automation 8 (4) (1992) 464–479.
[9] R.K. Miller, D.G. Stewart, H. Brockman, S.B. Skaar, A camera space
control system for an automated forklift, IEEE Transactions on Robotics
and Automation 10 (5) (1994) 710–716.
[10] J. Pagès, X. Armangué, J. Salvi, J. Freixenet, J. Martı́, A computer vision
system for autonomous forklift vehicles in industrial environments, in: In
Proc. of The 9th Mediterranean Conference on Control and Automation
MEDS’2001, 2001.
[11] A. Kelly, Automated Material Transport System,
http://www.rec.ri.cmu.edu/projects/amts/amts.shtml, 2003.
[12] AGV Products Homepage, http://www.agvp.com/.
[13] Egemin Automation Homepage, http://www.egeminusa.com/.
[14] FMC Technologies Homepage, http://www.fmcsgvs.com/.
[15] B. Jervis, Webb Company Homepage, http://www.jervisbwebb.com/.
[16] R. Pissard-Gibollet, P. Rives, Applying visual servoing techniques to
control a mobile hand–eye system, in: Proc. IEEE Intl. Conf. on Robotics
and Automation, 1995, pp. 166–171.
[17] R. Swain, M. Devy, Motion control using visual servoing and potential
fields for a rover-mounted manipulator, in: Proc. IEEE Intl. Conf. on
Robotics and Automation, 1999, pp. 2249–2254.
[18] D.C. MacKenzie, R.C. Arkin, Behavior-based mobile manipulations for
drum sampling, in: Proc. IEEE Intl. Conf. on Robotics and Automation,
1996, pp. 2389–2395.
[19] M. Seelinger, S.B. Skaar, M. Robinson, in: D.J. Kriegman, G.D. Hager,
A.S. Morse (Eds.), An alternative approach for image-plane control of
robots, in: Lecture Notes in Control and Information Sciences: The
Confluence of Vision and Control, Springer, London, 1998, pp. 41–65.
[20] D. Tsakiris, P. Rives, C. Samson, in: D.J. Kriegman, G.D. Hager,
A.S. Morse (Eds.), Extending Visual Servoing Techniques to Nonholo-
nomic Mobile Robots, in: Lecture Notes in Control and Information Sci-
ences: The Confluence of Vision and Control, Springer, London, 1998,
pp. 106–117.
[21] B. Horn, Robot Vision, MIT Press, Cambridge, 1986.
[22] R. Tsai, An efficient and accurate camera calibration technique for 3d
machine vision, in: Proc. of IEEE Conf. on Computer Vision and Pattern
Recognition, 1986, pp. 364–374.
[23] H. Zhuang, K. Wang, Z. Roth, Simultaneous calibration of a robot and
hand-mounted camera, IEEE Transactions on Robotics and Automation
11 (5) (1995) 649–660.
[24] S.M. Smith, J.M. Brady, Susan — a new approach to low level image
processing, International Journal of Computer Vision 23 (1) (1997) 45–78.
[25] D. Shin, S. Singh, J. Lee, Explicit path tracking by autonomous vehicles,
Robotica 10 (1992) 539–554.
[26] E. Baumgartner, S. Skaar, An autonomous vision-based mobile robot,
IEEE Transactions on Automotic Control 39 (3) (1994) 493–502.
1038 M. Seelinger, J.-D. Yoder / Robotics and Autonomous Systems 54 (2006) 1026–1038
[27] J.-D. Yoder, E. Baumgartner, S. Skaar, Reference path description for
an autonomous wheelchair, in: Proc. IEEE Intl. Conf. on Robotics and
Automation, 1994, pp. 2012–2017.
Michael Seelinger is a research engineer and vice
president of Yoder Software, Inc. He is also a
visiting instructor in the Department of Mechanical
and Industrial Engineering at the University of Illinois
Urbana-Champaign. He is the principal investigator
of the NASA-JPL Phase II SBIR contract that
supported the research and development of the MCSM
technology presented in this paper. In 1994 he
graduated from the University of Notre Dame with a
B.S. in mechanical engineering. He received his M.S. and Ph.D. in mechanical
engineering also from Notre Dame in 1996 and 1999 respectively. His research
interests include the visual control of both holonomic and non-holonomic robot
systems.
John-David Yoder is an assistant professor at Ohio
Northern University. He is also the president of
Yoder Software and was the principal investigator of
the NASA-JPL Phase I SBIR contract which began
development of the MCSM technology. He graduated
with a B.S. in mechanical engineering from the
University of Notre Dame in 1991. He received his
M.S. from Notre Dame in 1994 and the Ph.D. in 1996,
both in mechanical engineering.