Preprints of the Fourth IFAC

Symposium on Robot Control

19-21, 1994, Capri.
September 19-21,1994, Capri , Italy

SPACE ROBOTICS

G. HIRZINGER
German Aerospace Research Establishment (DLR)
Institute for Robotics and System Dynamics, Oberpfaffenhofen,
Germany, fax:
D-82234 Wessling, Germany, fax : +49
+49815328-1134, df28@master.df.op.dlr.de
8153 28-1134, email: dr28@master.df.op.dlr.de

Abstract. The paper tries to outline the state of the art in space robotics. It discus-
ses the technologies used in ROTEX,
ROTEX , the first remotely controlled space robot which
flew with shuttle COLUMBIA in April 93, 93 , but it also gives a review of the major
space robot projects envisioned by different nations for the next decade. The mecha-
tronics and dynamics aspects of space robots including free-flying systems are briefly
discussed . Broader attention is given to the telerobotic and teleoperational control
discussed.
loop structures, including predictive delay-compensating graphics simulation. The
paper finally tries to emphasize that by task level programming (involving" learning
(involving "learning
by showing" on a high level) future space robots will be powerful tools for scientists
and ground operators which are not robot specialists. The tele-sensor-programming
concept of ROTEX was a first step in this direction. We plead for flying a variety
of space robot systems in the near future in order to enhance experience with and
confidence in these technologies as quickly as possible.

Keywords. Space robotics, telerobotics, shared autonomy, tele-sensor-programming,

remote control

1 INTRODUCTION Real intuition and manual skill is particularly re-

After four decades of manned space flight,
flight , where
many activities have become routine, one might
forget that the space environment continues to be
extremely hostile to human beings. They have
to be encapsulated in vehicles (for intra-vehicular
special , extremely expensive
activities IVA) or special,
suits,, which protect them from the hazard of the
suits
space environment (for extra-vehicular activities
EVA).
EVA) . When comparing human skills with those
of present-day robots of course human beings in
general are by far superior, but when comparing
the skill of an astronaut in a clumsy space-suit
with that of the best available robot technology,
technology,
then the differences are already going to disap-
pear, the more if there is a remote control and mo-
nitoring capability on ground with arbitrarily high
computational and human brain power. For IVA
activities a robot basically would have to compare
with the full human skill and mobility; however to
be honest, many of the manual operations to be
done in a space-laboratory environment are fairly
simple standard operations, like handling parts,
opening and closing doors,
doors , pulling drawers, pus-
hing buttons etc. which have to be done just by
stepping through extensive, written procedures.
quested in non-nominal situations, e.g.
e.g. when a
tape recorder has to be repaired.
repaired . Although it is
not clear today when a multi-fingered robot hand
might be as skilled as the human hand and when
(if ever) a robot might show up real intelligence
and autonomy,
autonomy, it nevertheless is obvious that even
with today's technology and the available telero-
botic concepts based on close cooperation between
man (e.g.
(e.g. the ground operator) and machine there
are many tasks in space, where robots can replace
or at least augment human activities with reduced
cost at least from a long- term perspective.
Thus we are convinced that automation and ro-
botics (A&R) will become one of the most at-
tractive areas in space technology, it will al-
low for experiment-handling, inspection, mainte-
nance, assembly and servicing with a very limited
amount of highly expensive manned missions (es-
pecially reducing dangerous extravehicular activi-
ties).
ties) . The expectation of an extensive technology
transfer from space to earth seems to be much
more justified than in many other areas of space
technology.
This is one of the main
mam reasons why several ac-

695
tivities towards 8pace
space robotics have 8tarted
started in a big 8ucCess
success of ROTEX has given the 8pace
space robot
number of countries. A crude discrimination of community quite a stimulus. Nevertheless a few
envisaged robot systems are large arms for exter- big projects have been under preparation since a
nal servicing attached e.g.
e.g. to the space station,
station , couple of years meanwhile,
meanwhile , and they are briefly
smaller arms for internal servicing attached inside outlined in the sequel,
sequel , too.
too .
of laboratory modules and free-flying telerobot sy-
stems for assembly, inspection and repair. repair . One of
the largest projects by sure is the space station's
station 's 2.1 The space robot technology experiment
mobile servicing system (MSS) to be built by the ROTEX
Canadian space agency. NASA's own big robot
project,
project , the flight telerobotic servicer (FTS), was The experiment's main features were as follows
cut down some time ago apparently due to exces- (fig. 1):
1) :
sive development costs, but we are sure that the
use of robots will become a major issue for NASA small,, six-axis robot (working space,.."
• A small space 1
I"'oJ

again . Other remarkable mid-term

in the future again.
projects are the Japanese space station ""remote
manipulator system" (JEM-RMS) or the Japanese
ETS-7 project, an experimental flight telerobotic
servicer for maintenance and repair of space sy-
stems.
stems . While the shuttle manipulator arm, arm , which
has been flown with the space-shuttle a number
of times in the past (including spectacular actions
like the solar-max satellite rescue in 1984),
1984) , may be
seen as a crane-like predecessor system for the Ca-
nadian MSS
MSS,, there had been no space robot expe-
past . The European Space
rience in Europe in the past.
Agency ESA is going to prepare the use of robots
in the European part COLUMBUS of the space
station, replacing EVA's
EVA 's by use of the ERA large
manipulator (comparable to the JEM-RMS) and
augmenting internal activities by use of the AMTS
(Automatic Manipulation and Transport System).
Apparently those nations having less background
and history in manned spaceflight have the stron-
gest interest in space robotics.
robotics . We are convinced
that for complex, partly autonomous robots with
extensive ground control capabilities it would be
too risky to leap from zero experience to a fully
operational system;
systenl; therefore we have proposed in
1986 the space robot technology experiment RO-
TEX,
TEX , which has meanwhile successfully flown in
space (spacelab mission D2 on shuttle flight STS
55 from April 26 to May 6, 93). 93) . ROTEX contai-
ned as much sensor-based on-board autonomy as
possible from the present state of technology, but
on the other side assumed that for many years coo-
peration between man and machine via powerful
telerobotic structures will form the basis for high-
performance space robot systems operable especi-
ally from a ground station, too [1]. [1] .
2 SPACE ROBOT PROJECTS - A BRIEF
REVIEW
Let us refer to some of the most interesting space
robot projects mentioned in the introduction in a
little bit more detail. In general we think that the
m) flew inside a space-lab rack.rack . Its gripper
remote (fig. 2) was provided with a number of sen-
sors, especially two 6-axis force-torque wrist
sensors, tactile arrays,
arrays , grasping force control,
control ,
an array of 9 laser-range finders and a tiny
pair of stereo cameras to provide a stereo
image out of the gripper;
gripper; in addition a fixed
pair of cameras provided a stereo image of
the robot's
robot 's working area.
area.
• In order to demonstrate internal and exter-
nal servicing prototype capabilities three ba-
sic tasks were performed (fig. 2)
2)::
a) assembling a mechanical truss structure
from three identical cube-link parts
b) connecting/disconnecting an elec-
trical plug (orbit-replaceable-unit-ORU-
exchange using a "bayonet closure")
c) grasping a floating object, representa-
tive for grasping a floating satellite
• All operational modes which we see for the
coming years were verified III
in performing
tasks:
these tasks:
teleoperation on board (astronauts
using stereo-TV-monitor), see fig.
fig . 20.
teleoperation from ground (using predic-
tive computer graphics) via human ope-
rators and machine intelligence as well.
sensor-based tele-programming
tele-programming,, as well
as conventional off-line-programming.
The operational modes were based on a uni-
fied control approach for which we have coi-
ned the term tele-sensor-programming.
tele-sensor-programming.
• Typical goals of the experiment were:
were:
To demonstrate the performance of
a complex, multisensory robot system
with powerful man-machine-interfaces
(as are 3D-stereo-computergraphics, 6
dof control ball
ball,, stereo imaging),
imaging) , in a
variety of operational modes including
ground control.

696
 To verify self-calibration and joint con-
trol under zero gravity, as well as pg-
J.lg-
motion planning based on the requi-
rement that the robot's accelerations
while moving must not disturb any J.lg-
pg-
experiments nearby.
nearby.
- To demonstrate and verify the use
of sensorbased 6 dof-handcontrollers
balls")
("control balls ") under zero gravity.
Local sensory feedback from the multisensory
gripper as basis for "shared autonomy control",
as well as predictive delay-compensating graphics
simulations were key issues in the telerobotic con-
cepts, which made the experiment a big success.
We will refer to this topic later on.
on .
2.2 The MSS for Space Station
The Mobile Servicing System (MSS) for the Inter-
national Space Station will probably be the first
really operational space robotic system around the
year 2000. It will consist of a 7 degree-of-freedom
arm with a length of about 17 m (the Space Sta-
tion Remote Manipulator System SSRMS) and
a smaller dual arm robot, the Special Purpose
Dexterous Manipulator (SPDM). These two ma-
nipulator systems can work independently of each
other, or they can work together with the SPDM
attached to the end of the SSRMS (fig. (fig. 3).
3) . A
mobile transporter is supposed to move the MSS
along rails on the space station truss.
truss . The cap-
ture, manipulation and berthing of large payloads
pay loads
is performed by the SSRMS, while functions re-
quiring dexterous capabilities are satisfied by the
dual-arm SPDM, which will play a role in the
Space Station maintenance, assembly and in pay-
load servicing. It is capable to operate from fix-
tures of the mobile base system, from the end of
the SSRMS, or from fixtures on other structures
[43].
[43] .
All arms are similar in kinematics having clusters
of 3 joints at the shoulder and near the tip, with
a pitch joint at the elbow position near the mid-
point of each arm. The SSRMS is not only mova-
ble by the mobile transport system along rails,
rails , but
by its symmetrical arrangement of joints it has a
self-relocation capability and thus can step from
one power data grapple fixture to the next as dis-
tributed around the station, thus reaching areas
which cannot be reached by the mobile transpor-
ter. Different control modes are provided ranging
from manual single joint commands to full carte-
sian control (based on selectable reference frames)
and to the automatic execution of definable m<r mo-
tion sequences. Force-torque-sensors in the wrist
allow to close local sensory loops (force-moment
697
accommodation) following e.g. e.g. the resolved rate
control structures as proposed by Whitney [18]. [18] .
An important feature of the arm control capabili-'
capabili- '
ties is the Artificial Vision Function (AVF) which
processes the images from' MSS or Space Station
fromMSS
mounted cameras in real-time and outputs positi-
p08iti-
ons, orientations and velocities with respect to the
camera or a reference target in the field-of-view.
The outputs may be just displayed, or they may
be used e.g. to control the SSRMS or SPDM arm
in a feedback loop for tracking a target.
target .
The MSS manipulators basically will be control-
led by astronauts using separate non-force reflec-
ting hand- controllers for rotations and transla-
tion. Ground control was not provided originally,
but has been included recently in at least a basic
form .
form.
The Japanese JEM-RMS arm, supposed to per-
form external servicing at the Japanese space sta-
tion , is comparable to the MSS large manipula-
tion,
tor arm, but at its end it will optionally carry a
(single arm) small fine manipulator system.
system . Fi-
nally the European ERA arm designed for exter-
nal serving at the European COLUMBUS station
is in a similar way self-relocatable as mentioned
above with the SSRMS.
SSRMS . On the other side redun-
dancy and self-repair capabilities in the joints of
the MSS arms are unique to our knowledge but by
sure very important for future operational space
robot systems.
2.3 The AMTS project of ESA
Inside the pressurized module of the COLUMBUS
COL UMBUS
space station ESA plans to employ an automa-
tic infrastructural manipulation and transport sy-
stem AMTS supposed to release the astronauts
from routine handling tasks. The system compri-
sing a 7 dof manipulator will be mobile through
the laboratory by use of a 3 dof translational rail
system, so that in combination 10 degrees of fre-
edom are available and the robot may reach any
point in the lab. It is supposed to use an updated
version of the multisensory ROTEX gripper. An
appropriate robot control system called SPAll,CO
SPARCO
is under development presently guided by the Ita-
lian company TECHNOSPAZIO.
2.4 The ETS-7 project
ETS-7 is the 7th engineering test satellite of
NASDA (National Space Development Agency of
Japan) to develop and perform in-orbit experi-
ment of rendezvous and docking (RVD) and space
robot technologies. It presumably will be the first
free-flying telerobot system to be launched around
1997. There will be a chaser satellite with a force-
controllable 6 dof robot arm (and separate from
it on an own base an advanced robotic hand, as
developed by MITI and ETL), and a target satel-
lite (fig. 4). Automatic rendezvous and docking
experiments between chaser and target will be
augmented by a number of robotic experiments
aa are orbit replaceable unit exchange, simulated
u endeffectors envisaged will use force-torque wris1
fuel transfer, simulated battery exchange, target
handling.
satellite handling.
This last task would include that the robot grasps
the target (attached to the chaser), the docking
mechanism is released (the arm has to follow), and
then the arm moves the target around. Capture
of target satellite is still under consideration. The
chaser is 2,2 tons and the target 0,4 tons,
tons , while the
ORU is only 10 kg. kg. So the robot arm control sy-
stem must be robust for payload mass variations.
Special attention is paid to the cooperative con-
trol of satellite attitude and robot arm (see section
Ill).
Ill) . So one of the envisioned tasks is to identify
the coupled dynamics of the satellite mounted ro-
bot arm and the satellite with appendages such
as
u the solar panel and the antenna. Other tasks
are to verify feedforward control techniques, where
the estimated disturbance torques of the robot are
fed into the attitude control system, and to mon-
attitude
tion planing of the arm which minimizes .attitude
errors of the chaser.
By the way in the European community compara-
ble concepts toward free-flying robots have been
developed. The Italian SPIDER project e.g. has
a dual arm free-flying robot system in mind, while
the ESS (experimental serving satellite) studies of
the German space agency DARA (although in a
very premature state) aim at a free- flying robot
wh·ch
which might inspect (and possibly repair) one of
geoetationary satellites, which do not (or
the large geostationary
do no longer) work; a target example is the TV-
Sat1, a solar panel of which did not open, so it
Satl,
never became operable since launch in 1987.
3 MECHATRONICS AND DYNAMICS
ASPECTS
Design and control of space robot mechanisms is
Deaign
characterized by a number of special restrictions
technology.
poeed by space environment and technology.
Endeffectors are often designed as special tools,
e.g. so-called grapple fixtures (fig. 5) with own
latching or locking mechanisms e.g. for stepping
along space station structures, or for grasping
floating satellites (which of course should provide
the corresponding counterpart). Automatic tool
698
exchange mechanisms are important for enhan enhan·
ser·
cing versatility of these systems. For internal ser·
vicing we believe that two-finger grippers are ar
adequate solution, the more as in designing spac~
laboratories the parts to be handled (or e.g. do
ors and closures to be opened) may be provide<
with the appropriate interface that allows optiopti-
mal force-torque transmission. Most space robo!
robol
wris!
cameras. However sensor integration
sensors and cameras.
especially in free space with its temperature an<
aD<
radiation problems is fairly complicated and ex ex-
pensive.
pensive. Even inside laboratories space qualifica·
qualifica
tion requirements tend to hinder the use of advan·
advan
ROTE~
integration . Let us take ROTE]
ced electronics integration.
as an example.
With more than 1000 tiny SMD electronic anc aD<
several hundred mechanical components the RO·
TEX gripper is probably the most complex mul-
tisensory endeffector that has been built so far
The gripper was not space qualifiable on compo-
nent level especially because SMD technology ifi~
not yet generally permitted in space; so it hac
to undergo vibration, temperature, off-gasing ancan<i
so-calle~
EMC-tests as a whole; we finally got a so-callec
" waiver" , i.e. a special permission, from NASA
"waiver",
As a positive experience during the flight all grip-
per sensory and drive systems (which were per-
manently sending their temperatures down in ad-
dition to the measuring values) remained belo"belm'!
40°C, while being operable up to 80°C. All com-
ponents worked absolutely reliable.
We think that in the future articulated multi·
e.g. bio-
fingered hands will be useful in handling e.g.
logical experiments or in repairing systems, but
presently all mid-term projects aim at simpler en-
deffectors. Concerning space robot arms the sa-
ving of weight is a crucial item. Even in the RO-
TEX flight model fig. 6 (which was a comparabl~
comparably
small arm with 1 m range), weight saving in the
joints led to the effect, that the robot on ground
had to be Sll pended by strings for restricted test
suspended
operations. Large arms as the shuttle manipula-
tor normally are suspended by air bearings in flat
floor tests; but as an interesting fact the Cana-
dian MSS arm to our knowledge will not use these
kind of expensive and yet restricted test facilities,
but will fully rely on computer simulatioDB
simulations stu-
dies.
dies. Another interesting observation is, that even
in large arms like the shuttle manipulator, nearly
all of the elasticity that calls for active damping
control (not available in the shuttle manipulator)
originates from the joints, not from the arm struc-
tures. Again using ROTEX experience, one has to
take in account that backlash in space is perma-
nently active due to missing gravity preloading,
and that adaptive joint control using friction ob-
servers is very helpful. Different type of friction
observing joint controllers were uploaded during
the ROTEX mission.
We believe that space robotics may become a ma-
jor driver for a new,
new , extremely light-weight ro-r~
bot generation. For the astronaut training in RO-
TEX we developed a kinematically identical light-
weight version of the flight model, which was fully
operable on ground, and where we integrated new,
extremely compact joints with gear reduction 1 :
600 into carbon fibre grid structures [2], see fig. 7.
It shows up a 1:1
1: 1 ratio between own weight and
load capability (10 kg both). Presently a new ver-
sion of this arm is provided with torque-controlled
joints based on inductive torque sensing; for us
it leads into a new generation of multisensory,
torque-controlled (instead of position-controlled)
space robots.
In the final version all power and signal electronics
will be integrated in the arm using SMD minia-
electronics. The remaining problem will
turized electronics.
then be to qualify this new 7 dof arm for space, e.g.
be enveloping the link structures by appropriate
foils or moving into more special hybrid electr~
electro-
nics technology. By the way the arm is designed
nies
in a fully modular way and foldable to minimal
size for stowing it in a space-saving way.
A crucial issue in control of
offree-flying
free-flying robots is the
interaction between robot arm and carrier satel-
lite, as in contrast to a big shuttle or space station
the masses are in a comparable order of magni-
tude. In such a system the satellite's attitude and
poeition in general is controlled actively by m~
position mo-
mentum wheels and reaction jets (thrusters, which
are also used to desaturate the wheels after they
have absorbed disturbance torques for a while).while) .
An excellent survey of existing control strategies
is given in [6].
Briefly summarizing, four main methodologies
have been investigated so far:
far :
• the spacecraft's attitude and position remains
e.g. feeding the robot's reaction for-
fixed by e.g.
ces 1 (gained by computation) into the sa-
tellite attitude position control system, i.e.
providing feedforward compensation. Then
terrestrial robot control techniques may be
used, but excessive fuel consumption may be
the result.
• only the spacecraft's attitude is controlled by
feedforward compensation, not its translatio-
nal position; a kind of new kinematics has to
lIB
tin ,hi.
thia paper the tenn forces
t'orces when referring to
wrist load of a manipulator .tands
tthee wriat forces and
atanda for force. aDd
torques.
699
be developed as shown in [7].
• the spacecraft is not controlled (free-floating
system); of course this approach can only
be used in the absence of external forces
and torques. In [8] the existence of "dyna-
mic, path dependent singularities" has been
shown . When they are avoided, and the cor-
shown.
rect dynamic models are implied, then nearly
any terrestrial control is applicable [9], see
[10] .
also [10].
• the spacecraft's attitude and position con-
trol system is controlled in a coordinated way
to reach desired locations and orientations in
space, thus realizing an unlimited workspace
with a practically redundant manipulator.
In addition to these basic control categories as
proposed in the literature there are additional ef-
fects which are very difficult to simulate, e.g. im-
pact between the satellite-robot system and a tar-
get to be grasped. In [11] theoretical investigati-
ons and airbearing flat-floor experiments are de-
scribed which include these complicated mecha-
nIsms.
4 TELEROBOTIC CONTROL LOOPS
4.1
4.1 General remarks
teleoperati~
TOIn Sheridan, a pioneer in teleoperatio-
As Tom
nal systems, formulated [12], telerobots com-
bine the advantages of human remote con-
trol with the autonomy of industrial robots.
In the beginning of this kind of technology the
terms teleoperation and telemanipulation es-
sentially meant that an artificial antropomorphic
arm (a teleoperator or telemanipulator) is direc-
tly and remotely controlled by a human operator
just like an extension of his own arm and hand
[14], [25]. Typically such a teleoperator system
is supposed to work in hazardous and hostile en-
vironment [34], while the human operator in his
safe and remote control station may make full use
of his intelligence, especially when all relevant in-
formation is sent back to him from the remote
worksite of the teleoperator, e.g. TV-images, for-
ces and distances as sensed by the manipulator.
More specifically if a teleoperator is prepared to
repeat a task once shown by the operator and
especially if some local autonomy using sensory
feedback is provided rendering the arm even more
adaptive, we prefer to use the above-mentioned
term telerobot. Typically then the human ope-
rator serves as a supervisory master, not control-
ling the telerobot's every move, but issuing gross
commands refined by the robot or - in its ultimate
form - stating objectives which are executed vIa
the local robot sensors and computers.
In the following we will have a closer look into the
control structures of these telerobotic systems.
4.2
~. 2 Bilateral feedback concepts
Teleoperator systems are closely related to the
master slave concept, i.e. more or less indepen-
dent of the special realization there exist opera-
tional modes, where the manipulator slave arm
tries to track as precisely as possible the positional
(including rotational) or rate (i.e. velocity) com-
mands given by a human master who uses some
kind of an input device. In the early days of teleo-
peration this input device (called master arm)
was just a 1 : 1 replica of the slave arm, both
connected via mechanical coupling. More advan-
ced systems as presently used in nuclear power
plants show up pure electrical coupling between
the two arms, thus allowing remote control over
great distance provided that a TV-trans.mission
from slave worksite to the remote control station
is used.
used . An example for this concept of bilateral
position control is depicted in fig. fig . 8. By the
left hand side position control system the slave
joints (joint angle vector q,) are forced to follow
the master joints qm as closely as possible using
kind of a PD servo-controller with gain matrix S,
and damping matrix D.
The gain matrix S, may be derived from a desi-
red cartesian stiffness matrix Sr"
Sz " following Salis-
bury'8
bury 's stiffness control concept [13]:
(1)
relating slave joint errors ~q, and torques T, with
J, as the slave Jacobian.
J acobian.
The right hand side control system in fig. 8 provi-
des a safety- relevant kind of force reflection into
the master arm. Assume that the slave (due to
a master motion) moves into a wall or obstacle
not realized visually by the operator. The left
hand side control system - with the positional er-
ror increasing - would force the slave arm to exert
increasing forces (eventually arriving at the ma-
ximum forces exertable by the slave joints) and
thus might destroy the slave arm or the environ-
ment. The right hand side loop therefore feeds
back the positional joint errors (with a certain
=
gain Sm = J~Sz,mJm
J~Sr , mJm generating a correspon-
ding cartesian master stiffness Sr,m) into the ma-
ster arm joint motors. In an advanced system
700
these joint controls would in addition compensate
for the gravitational friction, inertial and coupling
forces using an inverse model (as indicated by the
compensating terms h,( q" q,) and hhm(qm,qm»
h,(q.,q.) m(qm, <tm))..
The operator in the ideal case would really sense
only the positional deviations caused e.g. by ex-
ternal forces and torqes (e.g. when colliding, lif-
ting loads etc.). In reality however such a system
provides reaction forces to the operator during any
kind of motion due to the unavoidable positional
p08itional
errors in such a servo system.
With these observations it comes clear that a sy-
stem as depicted in fig. 9 using an external wrist
force-torque sensor between the slave's last joint
and its endeffector is superior and capable of over-
coming these difficulties. The sensed external
f. at the slave's wrist has to be
force-torque vector f,
multiplied by J~,
J;;', the transpose Jacobian of the
master arm to yield the joint torques T Tmm,, which
are necessary to produce this same reaction force-
=
f m = f.
torque vector fm f, at the master's wrist [13].
[13] .
The right hand side positional feedback of fig. 8 is
no longer necessary; systems of this kind are called
bilateral force-reflecting master-slave systems.
The comments made above concerning inverse dy-
namic model techniques in the slave are still valid
here.
Note that the scheme in fig. 9 is cartesian based
now , i.e. the kinematics of master and slave are
now,
independent, provided that the master arm shows
up 6 degrees of freedom and has a similarly sha-
ped workspace (possibly down-scaled) compared
to the slave arm. The cartesian errors ~x, Lli ~x
are transformed via the inverse kinematics into
the corresponding joint (and if needed joint velo-
city) errors of the slave. 6-dof-hand controllers of
this type have been developed without and with
force-feedback (e.g. the kinesthetic handcon-
troller
troIler of Bejczy [14]). Although force-reflecting
hand controllers may provide a considerable per-
formance improvement [15], systems realized up
to now do not always show up the requested high
fidelity; this has partly to do with high feedb-
ack sampling rates needed (1 kHz seems a rea-
sonable value [16]), and with friction problems in
the hand-controller. In zero gravity (astronauts
as operators) no experience is yet available con-
cerning human's reaction to force reflection.
4.3
{3 Local autonomy concepts
In all cases discussed so far if the operator can-
not see the slave robot directly he may make use
of a TV-transmission line and look at a monitor
image (or better stereo TV image) instead of the
 8ignal transmission
real scenery. But if there are signal tran8mi88ion
delaY8 between master and slave
delays 8lave (e.g. when te-
leoperating a 8pace
space robot from ground) then the
bilateral schemes discussed so far implying the hu-
man arm in the feedback graphics tend to fail.
Predictive (i.e. delay-compensating) computation
and feedback of forces into the human arm seems
feasible when a perfect world model and a high-
fidelity force reflection device is available, but is
difficult to realize in practice.
Thus in the sequel we are addressing techniques
that do not provide any force-sensing in the hu-
man arm; these concepts are characterized by
local , artificially ge-
feedforward commands and local,
nerated compliance using impedance control
without a force sensor or active compliance
with local sensory feedback. Impedance control
means that the robot's endeffector reacts onto an
external force vector f just as a desired cartesian
impedance, i.e.
i.e. [17]
Mrx + Dr(xd - x) + Sr(Xd - x) = f (2)
where the mass, damping and stiffness matrices
Mzr , D rx , Sr
M Sx respectively are chosen arbitrarily and
Xd, Xd denotes a desired motion.
Xd,
Fig. 10 shows a corresponding structure [4] and
Fig.
indicates that feedback to the human operator is
now , the robot being locally compliant
only visual now,
with chosen impedance, so that it does not destroy
its environment when the artificial stiffness Sr
Sx is
chosen reasonably.
So far all the structures proposed are based on the
advanced concept of direct torque control on the
joint level. This however is not state of the art
until now, so it is justified to look for other prac-
tical concepts, especially using the disturbance re-
jecting positional command interfaces which are
robots . Active com-
offered by all present day robots.
pliance concepts based on local feedback of wrist
force sensor data into the positional interface go
back to Whitney (e.g. [18]) , and have led to
(e.g. [18]),
different implementation proposals for telerobots
[33]) .
(e.g. [16], [19], [20], [32], [33]).
The scheme proposed in [16], may be characteri-
zed by fig. 11, where the wrist forces are added
to the positional errors between master and slave
via a first order filter, again generating a cer~ain
certain
cartesian stiffness Sr
Sx and damping via the time
constant a (s here denotes the Laplace variable).
In the stationary case, e.g. when the master po-
sition XXm
m has been moved behind a rigid sur-
face in the environment XXenv enll (fig. 12), we have
ax = aXcompl
aXcompl so that the robot's motion stops.
stops.
701
In fact a scheme like that of fig. 11 works only if
the slave robot has some inherent mechanical com-
pliance SSRR either caused by the inevitable joint
compliance (leading to a position-dependent over-
all compliance) or by a dedicated compliance in
the wrist (e.g. Draper Lab's well-known remote
RCC) . Now if we ask for the
center compliance RCC).
resulting compliance S, relating X Xm
m - X enll to the
X env
sensed force f..
f. exerted by the slave, we have to
fig . 12)
solve the equations (see fig.
(x.. Xenll)SR =
(XI - Xenv)SR = SraXcompl
Sz.dxcompl =
--------------
~~
f. ----.....-.--
f.
Srax
~
=
Sz.dx = Sz(x
Sr(xmm -- x
x.)
.. ) (3)
f.
(4)
After a few elementary calculations we arrive at
(5)
i.e. an extremely stiff slave robot (SR very do-
Sr, . while a
minant) would lead to the desired Sx,.
very compliant slave would behave near to its na-
tural stiffness. Clearly by appropriate choice of
the stiffness matrix Sr
Sx one may generate different
compliance in different axes.
tWb concepts presented
Let us recall that the last twb
were based on artificial slave compliance without
and with a local force sensor feedback loop, and
no force reflection into the human arm. Basically
even in case of active local compliance force re-
flection into the human arm seems feasible, but
presumably it would be reasonable to supply force
feedback to the operator only in those directions
which are not locally force-controlled.
force-controlled .
Note that glove-like input devices as pure posi-
tional/rotational controllers and (force-reflecting)
too .
exosceletons fully fit into this framework, too.
Alternative concepts as developed at the German
Aerospace Research Establishment (DLR) and wi-
dely applied in ROTEX are based on local sen-
sory feedback and purely feedforward type of 6 dof
handcontrollers, too, but the master input devices
are designed in a way so that they work as rate
and force-torque command systems simulta-
neously in contrast to the positional master arms
discussed so far.
far . Motions permitted are very small
(typically 1 - 2 mm via springs, i.e. no joints) ma-
king the mechanical design fairly simple. DLR's
sensor or control ball (recently redesigned into
a "SPACE MOUSE" contains an optically mea-
suring 6 component force-torque sensor, the basic
principle of which is also used in DLR's compliant
wrist sensors [21].
The main features of the underlying more gene-
ral telerobotic concept are shown in fig. 13 and
fig. 14. Rudimentary commands dx dX (the former
deviations between master and slave arm) are de-
rived either from the sensor ball as forces (using
a sort of artificial stiffness relation dX =
dx = S; If)
or from a path generator connecting preprogram-
med points (dx being the difference between the
path generator's, i.e. "master's", position and the
commanded "slave" robot position x eom com ).). Due
to the above-mentioned artificial stiffness rela-
tion these commands are interpreted in a dual
way, i.e. in case of free robot motion they are
interpreted as pure translational/rotational com-
mands; however if the robot senses contact with
the environment, they are projected into the mu-
tually orthogonal "sensorcontrolled" (index f)
and" position-controlled" (index p) subspaces, fol-
lowing the constraint frame
frazne concept of Mason
[22]. These subspaces are generated by the robot
autonomously using a priori information about
the relevant phase of the task and actual sensory
information: to discern the different task phases
(e.g. in a peg-in-hole or assembly task) automa-
.tically.
tically. Of course the component dX dxpp projected
into the position controlled subspace is used to
feed the position controller; the component fJ f J pro-
sensor controlled subspace is eit-
jected into the sensorcontrolled
her compared with the sensed forces f,en,f .. n • to feed
(via the robot's cartesian stiffness S R) the ortho-
gonally complementary force control law, (which
in fact is another position controller yielding a ve-
locity xJ), or it is neglegted and replaced by some
nominal force vector f nom
nom to be kept constant e.g.
in case of contour following. We prefer to talk
about sensorcontrolled instead of force-controlled
subspaces, as non-tactile (e.g. distance) informa-
tion may be interpreted as pseudo-force informa-
tion easily, the more as we are using the robot's
positional interface anyway.
anyway. However we omit de-
tails as treated in [19] concerning transformations
between the different cartesian frames (e.g. hand
system, inertial system etc.).
The "resulting" stiffness (e.g. when the path ge-
nerator serves as master) in the sense of fig. 11
and the corresponding derivations equs.(3 - 5) are
the same as for the scheme of fig. 14 , given by
equ. (5).
It is worth to be pointed out again that in case
of real human teleoperation although there is no
force feedback into the operator's arm, the robot
702
using its local feedback exerts only the forces (may
be scaled) as given by the "teacher", thus is fully
under his control, or may behave autonomously in
predefinable sensor or position-controlled subspa-
ces. We are thus talking of shared control (see
[20], [32], [33]), i.e. some degrees of freedom are
directly controlled by a supervisor, while others
are controlled autonomously and locally by the
machine.
The local loops in fig. 13 are (at least presently)
characterized by modest intelligence but high
bandwidth, while the loops involving the human
band with but
operator's visual system are of lower bandwith
higher intelligence. Surely the long-term goal is
to shift more and more autonomy to the robot's
site and to move the operator's commands to an
increasingly higher level. Shared local autonomy
and feedback control as explained above using the
different type of gripper sensors was a key issue
for the success of ROTEX.
ROTEX .
4.4 Predictive control
When teleoperating a robot in a spacecraft from
ground or from another spacecraft so far away
that a relay satellite is necessary for communi-
cation, the delay times are the crucial problem.
Predictive computer graphics seems to be the only
way to overcome the main problems. Indeed pre-
dictive 3D-stereographic simulation was another
key issue for the success of the ROTEX experi-
ment, with its typical round-trip delays of 5-7 se-
conds. Of course for these kind of ideas to work
the same control structures and path planners had
to be realized on-board as well as in the predic-
tive graphics ground station. And this in turn
meant that not only the robot's free motion but
also its sensory perception and feedback behaviour
had to be realized in the" virtual environment" on
ground (fig. 15a).
Fig. 15 and fig. 16 are to outline that the human
operator at the remote workstation handles a 6
dof handcontroller
hand controller (e.g.
(e.g. control ball) by looking
at a "predicted" graphics model of the robot. The
control commands issued to this instantaneously
reacting robot simulator were sent to the remote
robot as well using the time-delayed transmission
links. In addition to the simulated preview the
ground-station computer and simulation system
may contain a model of the uplink and downlink
delay lines as well as a model of the actual states
of the real robot and its environment (especially
moving objects). The real down-link sensory and
joint data are compared with the down-link data
as expected from the simulator, the errors are used
in an "observer" or "estimator" type of scheme to
correct the model assumptions. Fig. 17 and fig. 18
indicate the excellent coindidence of laser range
data in ROTEX.
Overlaying graphically generated sceneries [23]
and real TV
TV-images
-images is particularly useful for cali-
bration, e.g. for assuring that the computational
viewpoint is identical to the real one, and that
the real world is close to the virtual one. Other
than that it may be sufficient in many cases to
overlay the predicted graphics with e.g. a wire
frame representation of the wrist as derived from
the down-link joint data or even only an indication
hand-frame. This was realized in ROTEX,
of the hand-frame.
where the real i.e. delayed robot gripper's posi-
tion was characterized just by a cross-bar and by
two patches displaying the gripper's actual ope-
ning width.
In ROTEX there was only one exception from the
local sensory feedback concept
concept.. It refers to image
processing. In the definition phase of ROTEX
(around 1986) no space qualifiable image proces-
sing hardware was available; nevertheless we took
this as a challenge for the experiment "catching
" catching
a free-floating object from ground" (fig. 15b). In
contrast to contact operations as necessary in case
of assembly we deal here with a nearly perfect
world model, as the dynamics of an object floa-
ting in zero g are well known. Hand-camera ·in-
formation on the free-flyer's pose (relative to the
gripper) was provided on ground using alterna-
tive schemes; one was based on the" dynamic vi-
the "dynamic
sion approach" as given in [3], using only one of
the two tiny hand-cameras, the other one was a
full stereo approach realized in a multitranspu-
ter system. In both cases the thus "measured"
object poses were compared with estimates as is-
sued by an extended Kalman filter that simula-
tes the up- and down-link delays as well as robot
(fig. 16); this Kalman filter
and free-flyer models (fig.
[5], [24], [41] predicts (and graphically displays)
the situation that will occur in the spacecraft af-
ter the up-link delay has elapsed and thus allows
to close the "grasp
" grasp loop" either purely operator
controlled, or via shared control
control,, or purely auto-
nomously.
nomously. Fig. 19 shows photos of the TV-scene
out of one of the hand cameras immediately before
successful,, fully automatic grasping from ground
successful
despite of 6 sec round-trip delay, following the
image processing approach in [3]. [3] . This automa-
tic, ground-controlled capture of the free-flyer was
one of the many spectacular actions of ROTEX ROTEX..
evertheless we have to state that this kind of
Nevertheless
delay-compensating control structure will not be
typical for future space robots,
robots , as local vision feed-
back will be feasible with on-board vision systems,
too.
too .
703
5 TOWARDS REMOTE TASK LEVEL
PROGRAMMING
From the experience we gained during ROTEX,
the" any time" quick switching between on-line te-
sensor based off-line programming
leoperation and sensorbased
(including later on-board execution), both based
uniformly on shared control,
control , is the most efficient
telerobotic concept for the coming years. In RO-
TEX a basic version of such a unified approach,
called tele-sensor-programming, was demon-
strated under the observation of a number of in-
ternational observers.
There are some similarities of this concept with
other approaches, e.g. the tele-programming
concept of R.
R . Paul and his coworkers [39], [40].
[40] .
In our approach any complex task like the dism<r
dismo-
unting and remounting of the bajonet cl08ure
closure is
assumed to be composed of elemental moves,
for which a certain constraint-frame- and een80r-
eensor-
type-configuration holds (to be selected by the
operator e.g. using a set of predefined alterna-
tives) . Thus automatic sensorbased path refine-
tives).
ment is clearly defined during these motion primi-
tives, which request various definitions and prode-
cures, in particular
- defining (or graphically demonstrating) the
nominal initial and goal situations (poeitioll8
(positions
or hand frames augmented by nominal een-
patterns) ; of course in case of on-line te-
sory patterns);
leoperation the gross-path in between is also aleo
given by the operator, else it is generated la-
ter (i.e.
(i .e. on-board) by the path planner.
- providing the a-priori knowledge on the C-
space configuration and the type of shared
control (active compliance or using nominal
sensory patterns).
- procedures for mapping sensory errors
errore into
positional/rotational errors (e.g. using & neu-
ral net training that allows to realize eenaor
8eD80r
fusion, too).
too) .
- procedures for mapping positional/rotational
errors into motion commands.
~oal
- procedures for recognizing actual and goal
states, thus determining e.g. the end of an
&0
move.
elemental move.
An impressive verification of these concepts was
as
given during the mission when the ground opera-
tor in on-line teleoperation stepped through the
ORU-exchange task without waiting at the end
of the elemental moves until the robot in space
had confirmed reaching the goal situation of the
corresponding elemental move.
move. In the same way
we were able to on-line teleoperate in the virtual
environment, but send up the gross commands at
some arbitrary time later, e.g. after finding a sa-
tisfactory motion.
But the crucial statement is, that the tele-sensor-
programming concept with its elemental move fea-
sensor based on-line teleoperation
tures comprises sensorbased
e.g. via predictive graphics simulation (including
local active compliance) as well as a correspon-
ding off-line-programming version, which may be
characterized as "sensorbased teaching by sho-
wing" . Hereby the robot is graphically guided
through the task (off-line on ground), storing the
relevant nominal situations (graphically simula-
ted) for later (e.g. associative) recall and refe-
rence in the on-board execution phase, after these
data packages have been sent up to the on-board
path planner. Indeed this mode of tele-sensor-
programming is a form of task-oriented, impli-
cit, off-line-programming which tries to overcome
the well-known problems of conventional approa-
ches, especially the fact that simulated and real
world are not identical. But instead of e.g. ca-
librating the robot (which is only half the story)
tele-sensor-programming provides the real ro-
bot with simulated sensory data that refer to rela-
tive positions between gripper and environment,
thus compensating for any kind of inaccuracies in
the absolute positions of robot and real world.
6 CONCLUSION
We believe and we hope that the successful flight
of ROTEX, the first remotely controlled robot in
space, has encouraged the robot community to
proceed straightforward towards designing semi-
autonomous space robots that relieve man from
monotonous or dangerous tasks. Time and tech-
nology is mature to fly a number of different sy-
stems during the next decade. To proceed quickly,
qualification standards, very conservative in gene-
ral, should be reviewed, so that it will possible to
lower development costs, fly more systems for the
same amount of money and gain broad experience
in short time.
time .
ROTEX also clearly showed that the information
and control structures in mission control centres
for future space robot applications should be im-
proved, allowing the robot operator on ground
permanent direct access to the different types of
uplinks and providing him with a continuous TV-
transmission link.
link .
Close cooperation between man (astronaut as in
fig . 20 or ground operator) and machine compri-
fig.
704
sing different levels of sensor-based robot auto-
nomy was important for ROTEX and will be cru-
cial for future space robots. It was clearly pro-
ven that a robot system configured
con figured in this flexible
way of arbitrary and fast switching between the
most different operational modes will be a power-
ful tool in assisting man in future space flight pro-
jects and it was impressively shown that even large
delays can be compensated by appropriate estima-
tion and predictive pre-simulation concepts.
For ongoing telerobotics research we presently see
the importance of two major directions for the
next years:
• to bring the sensor-based learning by sho-
wing programming ideas (as outlined before)
to such a high level that no longer a robot
expert is needed in order to remotely execute
arbitrary tasks in e.g. a robotized space- la-
boratory, but that the scientific experimenter
is sufficient for working with the robot system
• if presimulated and real sensory data show
major differences, let the system automati-
cally find out what was wrong (the real sensor
data or the simulated ones or the real robot's
motion or the environment model etc.).
Service robots (compared to industrial robots)
presently seem to become a major driver in ro-
botics technology progress. Space service ro-
bots in particular with the requirements for
light-weight, sensor-based autonomy and powerful
man-machine interfaces show up a lot of potenti-
potent i-
als for terrestrial spin-off (see e.g. [42]).
[42)).
REFERENCES
[1] G. Hirzinger, B. Brunner, J. Dietrich, J. Heindl;
"ROTEX - The First Remotely Controlled
Robot in Space", IEEE Int. Conference on
Robotics and Automation, May 8-13, 1994,
San Diego, California.
[2] G. Hirzinger, A. Baader, R. Koeppe, M. Schedl,
"Towards a new generation of multisensory
light-weight robots with learning capabili-
ties" , IFAC'93 World Congress, Sydney, Aus-
tralia, July 18-23, 1993.
[3] D. Dickmanns, "4D-dynamic scene analy-
sis with integral spatio-temporal models",
Fourth Int. Symposium on Roboti~
Robotics Rese-
arch, Santa Cruz, Aug.
Aug. 1987.
[4] G. Hirzinger, J. Heindl, K. Landzettel, "Con-
trol Structures in Sensor-Based Telerobotic
Systems"
Systems",, ICAR 5th Int. Conf. on Advanced
Robotics, Pisa, June 1991.
[5] Christian Fagerer and Gerhard Hirzinger, "Pre-
dictive Telerobotic Concept for Grasping a
Object" , Proc. IFAC Workshop on
Floating Object",
Spacecraft Automation and On-Board Auto-
nomous Mission Control, Darmstadt, Sept.
1992
[6] St. Dubowsky, E. E. Papadopoul08, "The Kine-
matics, Dynamics, and Control of Fee-Flying
and Free-Floating Space Robotic Systems",
IEEE Transactions on Robotics and Automa-
tion (Special Issue on Space Robotics), Vol.
9, No. 5, pp. 531-543, October 1993.
R . Longman, R. Lindberg, M. Zedd, "Satellite-
[7] R.
mounted robot manipulators - New kine-
matics and reaction moment compensation" ,
Int. J. Robotics Res. vol. 6, no. 3, pp. 87-103,
Fall 1987.
[8] E.
E. Papadopoulos, S. Dubowsky, "Dynamic sin-
gularities in the control of free-floating space
manipulators",, ASME J .Dyn.Syst
manipulators" .Meas.,
.Dyn.Syst.Meas.,
Contr., Vo1115,
Vol115, No. 1, pp. 44-52, March 1993.
E. Papadopoulos, S. Dubowsky, "On the nature
[9] E.
of control algorithms for free-floating space
manipulators" , IEEE Trans. Robotics Auto-
mat. Vol. 7, pp. 750-758, Dec. 1991.
[10] Y.
Y. Umetani and K. Yoshida, "Resolved mo-
tion rate control of space manipulators with
J acobian matrix", IEEE Trans.
generalized Jacobian Trans.
on Robotics and Automation,
Automation , Vol. 5, No. 3,
1989.
[11] K. Yoshida, "Space Robotics Research Ac-
tivity with Experimental Freefloating Robot
Satellite (Efforts) Simulators", Third Inter-
national Symposium on Experimental Robo-
tics, Kyoto, Oct. 1993.
[12] T.B. Sheridan, "Merging Mind and Machine",
Technology Review, 33-40, Oct. 1989.
[13] J.J. Craig, Introduction to Robotics. Addison-
Wesley Publishing Company, ISBN 0-201-
10326-5, 1986.
[14] A.K. Bejczy, "Robots as Man-Extension Sy-
stems in Space",
Space" , Proceedings IFAC Congress.
Budapest, 1984.
[15] B. Hannaford, "Stability and Performance
Trade-offs in Bilateral Telemanipulation",
Telemanipulation" ,
Proceedings IEEE Conference Robotics and
Automation,
Automation , Scottsdale, 1989.
[16] W.S. Kim, B. Hannaford, A.K. Bejczy,
"Shared Compliance Control for Time-
Delayed Telemanipulation", Proceedings IS-
MCR Symposium on Measurement and Con-
trol in Robotics, Houston, Texas, 1990.
[17] T. Yoshikawa, Foundations of Robotics.
Robotics. MIT
Press, ISBN 0-262-24028-9, 1990.
[18] D.E. Whitney, "Force Feedback Control of
Manipulator Fine Motions". Journal of Dy-
namic Systems, Masurement and Control, 91-
97, 1977.
705
"Se08Ory feedb-
[19] G. Hirzinger, K. Landzettel, "Sensory
ac~ structures for robots with supervised lear-
aclt
ning" , Proceedings IEEE Conference Robo-
ning",
tics and Automation, S. 627-635, St. Louis,Louia,
Missiouri, 1985.
"Desisn and
[20] S. Hayati, S.T. Venkataraman, "Desi«n
Implementation of a Robot Control Sy,ternSy.tern
with Tra.ded
Traded and Shared Control Capability" ,
Proceedings IEEE Conference Robotic. and
Automation, Scottsdale, 1989.
[21] G. Hirzinger, J. Dietrich, "Multisensory r0-
sensor based path'
bots and sen80rbased path ·generation", Pro-
ceedings IEEE Conference Robotics and Au-
tomation, San Francisco, 1986.
[22] M.T. Mason, "Compliance and force control
for computer controlled manipulators". Pro-
ceedings IEEE Trans.
Trans. on Systems,
Systems; Man and
Cybernetics, Vol SMC-11, No 6, pp.418-432,
1981.
[23] A.K. Bejczy, W.S. Kim, St.C. Venema, "The
Robot: Predictive Displays for Te-
Phantom Robot:
leoperation with Time Delay", Proceedinp
IEEE Conference Robotics and Automation,
Cincinnati, 1990.
[24] G. Hirzinger, J.J . Heindl, K. Landzettel, "Pre-
dictive and knowledge-based telerobotic con-
concepts".. Proceedings IEEE Conference
trol concepts"
Robotics and Automation, Scottsdale, Ari-
zona, 1989.
[25] L. Conway, R. Volz, M. Walker, "Tele-
Systems: Methods and Archi-
Autonomous Systems:
tectures for Intermingling Autonom01l8
Autonomoua and
T~lerobotic
Telerobotic Technology", Proceedings IEEE
Conference Robotics and Automation, a.. b
leigh, 1987.
[26] P. Simkens, J.J . De Schutter, H..
H. Van Bru_el,
BrUMel,
"Force/Torque Sensor Emulation in a Com-
pliant Motion Simulation System", Procee-
dings of CSME Mechanical Engineerinl
Engineerin, Fo-
rum, Toronto, 1990.
[27] G. Hirzinger, J. Dietrich, B. Brunner, "Mul-
Proceedinp
tisensory telerobotic concepts", Proceedings
IEEE Workshop on Intelligent Motion Con-
trol, Istanbul, 1990.
[28] S. Hayati, T. Lee, K. T80, P. Backes , " A teet-
K. Tso, teat-
bed for a unified teleoperated- autonOlDOUB
autonomoua
dual-arm robotic system" . Proceedings IEEE
Conference Robotics and Automation, Vol 2,
1090-1095. Cincinnati, Ohio, 1990.
[29] T.B.
T .B. Sheridan, "Human supervisory controlCODtrol
of robot systems". Proceedings IEEE Confe-
rence Robotics and Automation, San Fran-
cisco, 1986.
[30] A.K. Bejczy, "Smart sensors for smart hands"
presented at the AIAA-NASA Conference on
"Smart" Remote SeD80r8,
Se08Ors, AlAA
AIAA paper No.
78-1712, Hampton, Virginia, 1978.
[31] J.J.
J .J. Craig, M.H. Raibert, "Hybrid Posi- [43] C.P. Trudel, D.G
D.G.. Hunter, M.E. Stieber, "Con-
tion/Force Control of Manipulators"
Manipulators".. Tran- trol and Operation of Space Manipulator Sy-
AS~1E, Journal of Dynamic Sy-
saction of the ASME, stems", NATO-AGARD Lecture Series 193,
stems, Measurement and Control, Vol. 102, Advanced Guidance and Control Aspects in
(6/ 1982) 126-132.
(6/1982) Robotics, June 1994.
[32] P.G. Backes, K.S. Tso, "UMI: An Interactive
Supervisory and Shared Control System for
Telerobotics", Proceedings IEEE Conference
Robotics and Automation, Cincinnati, 1990.
[33] W.S. Kim, B. Hannaford, A.K. Bejczy, "Force telemonipulction processOt"
elemon'pulotion processor

Reflection and Shared Compliant Control in

Operating Telemanipulators with Time De-
lay" , IEEE Trans.
lay", Trans. on Robotics and Automa-
tion, Vol. 8, No.No. 2, 1992. G
R.. Lumia, "Space Robotics:
[34] R Automata '---_. . .F
.
in Unstructured Environment", Proceedings (Oom1..-'-monioulotor )

IEEE Conference Robotics and Autmoma-

tion, Scottsdale, 1989.
ground
K.. Kosuge, J. Ishikawa, K. Furuta, M.
[35] K Sakai,,
rvl. Sakai
"Control of Single-Master Multi-Slave l\la- Ma-
nipulator System Using Vim", Proceedings ROTEX
IEEE Conference Robotics and Automation,
Cinctnnati, 1990.
Cincrnnati,
[36] S. Lee, G G.. Bekey, A.K.
A.K. Bejczy, "Computer
Control of Space-Borne Teleoperators with
Sensory Feedback", Proceedings IEEE Con-
ference Robotics and Automation
Automation,, St. Louis,
Louis,
l\lissiouri,
Missiouri, 1985.
T.B.
[37] T .B. Sheridan, "Space Teleoperation through
Time Delay: Review and Prognosis", IEEE
Transactions on Robotics and Automation
Automation::
Special Issue on Space Robotics, Vol. 9, No.
pp.. 592-606, Oct. 1993.
5, pp
W.S . Kim, A.K. Bejczy, "Demonstration of
[38] W.S.
a High Fidelity Predictive/Display Techni-
que for Telerobotic Servicing in Space" , IEEE
Transactions on Robotics and Automation:
Special Issue on Space Robotics, Vol. 9, No. No.
5, pp. Oct.. 1993.
pp . 698-702 Oct
[39] J. Funda, R .P. Paul, "Efficient control of a r~
R.P. ro-
botic system for time-delayed environments" .
Proceedings of the Fifth International Con-
ference on Robotics and Automation, pages
133-137, 1989.
[40] M.R. Stein, R R.P.
.P. Paul, "Operator interac-
tion for time-delayed teleoperation with a
behaviour-based controller" , IEEE Conf. R~ Ro-
boties
botics and Automation, San Diego, 1994.
G.. Hirzinger, K.
[41] G K. Landzettel
Landzettel,, Ch
Ch.. Fagerer, "Te-
lerobotics with large time delays - the RO-
TEX experience", IROS'94IROS'94,, Int. Conf. on In-
1. Schematic representation of ROTEX
Fig. 1.
Systems, ~lunich,
telligent Robots and Systen1s, Sept..
Munich, Sept
1994.
1994 . (upper part) and integration in spacelab
(picture, courtesy of DORNIER)
[42] G. IIirzinger, Gon1bert, J. Dietrich, JJ.. Shi,
Hirzinger , B. Gombert,
"Transferring space robot technologies into
terrestrial applications" ,25th Symp~
, 25th Intern. Sympo-
si urn on Industrial Robots, Hannover, April
sium
1994..
1994

706
 Fig. 2. ROTEX experiment set-up in DLR
laboratory, where the multisensory
gripper is below the three-part truss
structure just in front of the bayonet
closure (the "ORU")

Spec'-' Puf.-._
Sp.c'-ol,.."".o..
O' .... OUI
OeI1.. OU.~I.,
LIMMpuIa,..
(SPOt.4
(SPO,,!
(SSAUS)

"0..,0.,.
"o-.lO.'a
Of.,.. F.......
O,aPP'e ,........ (4)
,POOF.)
(POOFt•

...OS
Came,.

Fig. 3. Mobile Servicing Centre with SPDM

(with courtesy of NASA/CSA)

707
 Orbital R lacement
Ilc:ement Unit (ORU)

Ad.load Robotic
Advanced Robotic Hand (M I1VE11..)
HInd (MrIlIE11..\

Monitor Camera
\ Monitor Camen
Camera

Dc loyab1e Truss eN AL)

Fig. 4. ETS-7 Onboard Robot Experiment Payloads

Pay loads

Fig. 5. Latching End Effector and Power Data Grapple Fixture (courtesy CSA)

708
Fig.
Fig . 6. ROTEX-flight robot built by DORNIER (courtesy of DORNIER)
6. ROTEX-ftight

Fig.
Fig . 7.
7. DLR-light-weight robot for astronaut training

709
 Fig. 8. An advanced bilateral position feedback master-slave teleoperator system

\
\
\
\
\
\
\
\
" ", \
\
-------~} master
(->-----~ joints
...............
..........
~=="-,

Fig.
Fig. 9.
9. A bilateral force feedback system, when master and slave are kinematically different
1
Iteroo-TV
IItoroo-1Y .IJ monitor

d~-------~~---~
d,.------$-r------~

Fig.
Fig . 10.
10 .

Impedance control yields a compliant slave without using a force sensor, transmission delay line indicated

710
 stereo-
.tereo-TV
TV I

wrist sensor

\
~ I
I
\ I
\ I
\ I
" \ I
"" \ I
----~\ I
.." I
,..."..".." ..... I
..-A==~ I
I
I
6x I
q I
I
I
I

XI

Fig. 11: Active compliance teleoperation via the positional slave interface (transmission delays indica.ted,
indicated,
master compensation not drawn)

6x
~
I
I
I
I
I
~ .... I

I
I F ig. 12. Active compliance causes the slave
Fig.
~;v: robot to be commanded stationarily
I
I somewhat "behind" a rigid surface Zen"
Zen",
in the environment, the final position
z, not beeing identical to the master
position Zm

fine path generation gross path generation

I
I pr09rommln9.
programming.
I

__+
__ --i-- __
on on
Otl menue-control
menue-control
c:ornero. board
boord I 9round
ground stereo-
s tereo- TV

~-':OPhICD
~-:o~.~

: r----
r----~
: IJ
: IJ
I
I
IJ
gross I I
commands I II
~---_..&.._ ..... -~-,
~
:

Fig. 13: Overall loop structures for the sensor-based telerobotic concept of ROTEX

711
 .---_:>-----.; f + rObOt
J+ {~~~~~iC.S, 1-_______x_R___-,
dynamics,
x
R

kinematics
q
com
com I
I
I
I
I
I potentiolly
potentially
tost
I fast
force loop
I torco
I gain oprlorl
I r--- Information
I ,-and Cp
on C ,-ond tosk Info
task
I
II q"com
q"eom
I
I
I 90noroto C
generate
I project 6x f-6)($.
I into C,andC p 6x
L_ force
control x com
low
xf,com f,
position 6xp
6x p
control I-----'-----------------~
xp,c om
p,com low
low

Fig. 14: A local closed loop concept with automatic generation of force
and position controllend directions and artificial robot stiffness

spacecraft spacecraft
reol world reol world
r-----------,
r----------..,

r-~--j L+-------.
I .. sensor-based on
!i ~-----t--~r_------__,
I:_-_-_f
L_..t
L_~
r- --.

L_"""""1 path refinement board on

I
board
III: sensor
reol molion and
real motion
dolo
data
IIreol mollon and
l: sensor doto
dolo

on on
ground
ground

world model update

gross
9 fOSS
commands
commonds
~
b_----,-<~__m_comm____.Qnd'
9'oSS

slereogfophlc s imul ation of

stert09fophle simulation
stereogrophic simulation of robot. environment,
en vironment. sensory
robot, environmen t, sensory
en,":ironment. perception and polh
path refinement do' hand
6 dot hond controller
controll er
percepllon° and path refinement
percepllon 6 dot hand controller or poth
path plonner
planner
or path planner

Fig. 15: Presimulation of sensory perception and path refinement

in case of teleoperation from ground
a) local on-board sensory feedback (e.g. tactile contact)
b) sensory feedback via groundstation (grasping a free-flyer)

712
 ·poce
grollnd
!
----------,------
:
rr ". . I

I
""q..
O"!/o
~----------------
#/~,..-
.~
I 0#.."
I
I r---------,
r ---------l
i
• "leri IwMwltlll,.
I
I
I
I
predlcl.d m...utld oulput I
m,.,,,,,e4 o"tpt.tl
(d.loy.d)
(d .. .,.d) I
trophic.
down-link nUm.lod .' ----,
- --I
modol
oulpul II I
II I
,obol!
.con.
...... tlon
nlimolion I I
~c.:~'c:::o:.~ er
I I
rI "
.,J
II
I
II
.up.,po.iUon~
I I I
I ~
<on""
conVoI I
I1
I
••p"p ....... '-LI J...I .lIoch 1
i~ I -----r--I
.11 •• 1.
• 1 nUmot..
co,ucUon • 0'
co,rectlonl .. limot .. I
I I I I I
I I rI I I
I I L_~t~'.!'~h1~~~
II L-
: : I
IL ______________________d~'.!_d..!r!!N~~c~. _ _ _ _ _ _ _ _ JI
J ~~~X~~~~
: I
II
L ________________________________ ~~~~~
~~~~~ ___ J ~

Fig. 16. Block structure of predictive estimation

scheme

Fig.
Fig. 17.

Sensorsimulation:
Sensorsimulation: Range finder
simulation in the" virtual" workcell
the "virtual"
environment.
environment . In addition to the 5
simulated rays out of the gripper the
bars in the right lower corner indicate
the same simulated (bright) and the
corresponding real (dark) range values
as registrated by the real robot.
robot .

ORU C .. cl .o .... O.

Fig.
Fig. 18.

Correlation between presimulated (for

comparison delayed) and real sensory
data (in closed loop each) was nearly
t
11
'.07'
•. eH
l .... (·50 ... . . eJ
(·~ n ..... c)
'.07.
', 0"
(10 • • ) '.07. perfect in ROTEX.
ROTEX . These recordings of
(·50..... e J (10 • • )
ORU £ .. ehoMg_ ORU
QAU £C .. ehong.
c hono_
the four finger-range-finders pointing
"downwards" were made during
sensorbased teleoperation
teieoperation when
'I ,fi
removing from the ORU bayonet
~

. closure (fig.
(fig. 17).
17) .
I
!

!! iI
.'i

I
'.0""
1.0'" ,.ey.
• • •• •
' .e"
'.07' . . •••
'.OP.
ttlI....
... . { .·~
[ 50..... cJJ
O ... . . e (1 0 •
(10 ."• ) tt I....
l .... (·50
( . 50..... oJ
..... 0) (10 •
(la ."•)

713
 Fig. 19.
Two subsequent TV-images out of one
of the hand cameras shortly before
grasping the free flyer automatically
from ground. The dark areas at the left
and right lower part are the gripper
jaws.
Jaws.

. ....

Fig. 20:
Fig. 20 : Payload specialist Hans Schlegel tele-operating the robot via 8tereO di8play and the 6 dof
stereO TV display
handcontroller (courtesy NASA)
ASA)

714