You are on page 1of 12

A MOBIUS AUTOMATON: AN APPLICATION

OF ARTIFICIAL INTELLIGENCE TECHNIQUES


by
N i l s J . Nilsson
Stanford Research I n s t i t u t e
Menlo Park, C a l i f o r n i a

Summary Several e a r l i e r reports' 1 " and papers2- 4 de-


s c r i b i n g the p r o j e c t have been w r i t t e n ; in t h i s
A research p r o j e c t applying a r t i f i c i a l i n t e l - paper we s h a l l describe i t s status as of e a r l y
ligence techniques to the development of i n t e - 1969 and discuss some of our f u t u r e p l a n s .
grated robot systems Is d e s c r i b e d . The experimen-
t a l f a c i l i t y c o n s i s t s of an SDS-940 computer and The robot v e h i c l e i t s e l f is shown in F i g . 1.
associated programs c o n t r o l l i n g a wheeled v e h i c l e It is propelled by two stepping motors independent-
t h a t c a r r i e s a TV camera and other sensors. The ly d r i v i n g a wheel on e i t h e r side of the v e h i c l e .
primary emphasis is on the development of a system It c a r r i e s a v i d i c o n t e l e v i s i o n camera and o p t i c a l
of programs for processing sensory data from the range-finder in a movable " h e a d . " Control l o g i c on
v e h i c l e , f o r s t o r i n g relevant i n f o r m a t i o n about board the v e h i c l e routes commands from the computer
the environment, and f o r planning the sequence of to the appropriate a c t i o n s i t e s on the v e h i c l e .
motor a c t i o n s necessary to accomplish tasks in the In a d d i t i o n to the d r i v e motors, there are motors
environment. A t y p i c a l task performed by our to c o n t r o l the camera focus and i r i s s e t t i n g s and
present system r e q u i r e s the robot v e h i c l e to r e - the t i l t angle of the head. (A motor to pan the
arrange (by pushing) simple objects in i t s e n v i r o n - head is not yet used by present programs.) Other
ment . computer commands arm or disarm i n t e r r u p t l o g i c ,
c o n t r o l power switches, and request readings of
A novel f e a t u r e of our approach is the use of a the status of various r e g i s t e r s on the v e h i c l e .
formal theorem-proving system to plan the execution Besides the t e l e v i s i o n camera and r a n g e - f i n d e r
of h i g h - l e v e l functions as a sequence of o t h e r , sensors, several " c a t - w h i s k e r " touch-sensors are
perhaps l o w e r - l e v e l , f u n c t i o n s . The execution of attached to the v e h i c l e ' s perimeter. These touch
these, i n t u r n , requires a d d i t i o n a l planning a t sensors enable the v e h i c l e to know when it bumps
lower l e v e l s . The main theme of the research is i n t o something. Commands from the SDS-940 computer
the i n t e g r a t i o n of the necessary planning systems, to the v e h i c l e and i n f o r m a t i o n from the v e h i c l e to
models of the w o r l d , and sensory processing sys- the computer are sent over two s p e c i a l radio l i n k s ,
tems i n t o an e f i i c i e n t whole capable of performing one f o r narrow-band t e l e m e t e r i n g and one f o r t r a n s -
a wide range of tasks in a r e a l environment. mission of the TV video from the v e h i c l e to the
computer.
Key Words
The purpose of our robot research at SRI is to
Robot study processes f o r the r e a l - t i m e c o n t r o l of a
Robot System robot system that i n t e r a c t s w i t h a complex e n v i r o n -
V i s u a l Processing ment. We want the v e h i c l e to be able to perform
Problem-Solving various tasks t h a t r e q u i r e it to move about in i t s
Question-Answering environment or to rearrange o b j e c t s . In order to
Theorem-Proving accomplish a wide v a r i e t y of tasks r a t h e r than a
Models of the World few s p e c i f i c ones, a robot system must have very
Planning general methods. What is required is the i n t e g r a -
Scene A n a l y s i s t i o n in one system of many of the a b i l i t i e s t h a t
Mobile Automaton are usually found separately i n i n d i v i d u a l A r t i f i -
c i a l I n t e l l i g e n c e programs.
Acknowledgment
We can group most of the needed a b i l i t i e s i n t o
At l e a s t two dozen persons at the Stanford three broad classes: (1) p r o b l e m - s o l v i n g , (2) mod-
Research I n s t i t u t e have made s u b s t a n t i a l c o n t r i b u - e l l i n g , and (3) p e r c e p t i o n :
t i o n s to the p r o j e c t that tne author has the good
f o r t u n e t o describe i n t h i s paper. A l l o f u s ex- (1) Problem-Solving
press our a p p r e c i a t i o n to the Rome A i r Development
Center and the Advanced Research P r o j e c t s Agency, A robot system accomplishes the tasks given it
who supported t h i s research under Contract No. by performing a sequence of p r i m i t i v e a c t i o n s , such
F 30602-69-C-0056. as wheel motions and camera readings. For e f f i -
ciency, a task should f i r s t be analyzed i n t o a
I Introduction sequence of p r i m i t i v e actions c a l c u l a t e d to have
the desired e f f e c t . This process of task a n a l y s i s
At the Stanford Research I n s t i t u t e we are im- i s o f t e n c a l l e d p l a n n i n g , because i t i s accom-
plementing a f a c i l i t y f o r the experimental study of p l i s h e d before the robot begins to a c t . Obviously,
robot systems. The f a c i l i t y c o n s i s t s of a t i m e - in order to p l a n , a robot system must "know" about
shared SDS-940 computer, several core-loads of p r o - the e f f e c t s o f i t s a c t i o n s .
grams, a robot v e h i c l e , and s p e c i a l i n t e r f a c e
equipment.

-509-
(2) Modelling II SOME SPECIFIC FUNCTIONS OF THE ROBOT
SYSTEM AND THEIR ASSOCIATED PROBLEM-
A body o f k n o w l e d g e a b o u t t h e e f f e c t s o f a c - SOLVING PROCESSES AND MODELS
t i o n s i s a type o f model o f t h e w o r l d . A robot
p r o b l e m - s o l v i n g system uses t h e i n f o r m a t i o n s t o r e d A. Low L e v e l F u n c t i o n s
i n t h e m o d e l t o c a l c u l a t e what sequence o f a c t i o n s
w i l l cause t h e w o r l d t o b e i n a d e s i r e d s t a t e . As The r o b o t s y s t e m i s c a p a b l e o f e x e c u t i n g a
t h e w o r l d c h a n g e s , e i t h e r b y t h e r o b o t ' s own a c - number o f f u n c t i o n s t h a t v a r y i n c o m p l e x i t y f r o m
t i o n s o r f o r o t h e r r e a s o n s , t h e m o d e l must b e u p - the simple a b i l i t y t o t u r n the d r i v e wheels a c e r -
dated to record these changes. A l s o , new i n f o r m a - t a i n number o f s t e p s t o t h e a b i l i t y t o c o l l e c t a
t i o n l e a r n e d a b o u t t h e w o r l d s h o u l d b e added t o number of b o x e s by p u s h i n g them to a common a r e a
the model. of the room. The o r g a n i z a t i o n o f t h e s e f u n c t i o n a l
a c t i o n - u n i t s i s not s t r i c t l y h i e r a r c h i c a l , a l -
(3) Perception though f o r d e s c r i p t i v e convenience we w i l l d i v i d e
them i n t o two c l a s s e s : l o w l e v e l and h i g h l e v e l
Sensors are necessary to g i v e a r o b o t system functions.
new i n f o r m a t i o n a b o u t t h e w o r l d . B y f a r t h e most
i m p o r t a n t sensory system i s v i s i o n , s i n c e i t a l - Of the f u n c t i o n s that we s h a l l mention h e r e ,
l o w s d i r e c t p e r c e p t i o n o f a good s i z e d p i e c e o f the s i m p l e s t are c e r t a i n p r i m i t i v e assembly l a n -
t h e w o r l d beyond t h e r a n g e o f t o u c h . Since we a s - guage r o u t i n e s f o r m o v i n g t h e w h e e l s , t i l t i n g t h e
sume t h a t a r o b o t s y s t e m w i l l n o t a l w a y s have h e a d , r e a d i n g a TV p i c t u r e , and so o n . Two e x -
s t o r e d i n i t s model e v e r y d e t a i l o f t h e exact c o n - amples of t h e s e a r e MOVE and TURN; MOVE c a u s e s t h e
f i g u r a t i o n o f i t s w o r l d and t h u s c a n n o t know p r e - v e h i c l e to r o l l in a s t r a i g h t l i n e by t u r n i n g both
c i s e l y the e f f e c t s o f i t s every a c t i o n , i t also d r i v e w h e e l s i n u n i s o n , and TURN c a u s e s t h e v e h i c l e
needs s e n s o r s w i t h w h i c h t o c h e c k p r e d i c t e d c o n - t o r o t a t e about i t s c e n t e r b y t u r n i n g t h e d r i v e
sequences a g a i n s t r e a l i t y a s i t e x e c u t e s I t s p l a n s . wheels in o p p o s i t e d i r e c t i o n s . The a r g u m e n t s o f
MOVE and TURN a r e t h e number of s t e p s t h a t t h e
The i n t e g r a t i o n o f s u c h a b i l i t i e s i n t o a d r i v e w h e e l s a r e t o t u r n (each s t e p r e s u l t i n g i n a
s m o o t h l y - r u n n i n g , e f f i c i e n t system p r e s e n t s b o t h v e h i c l e m o t i o n o f 1/32 i n c h ) and " s t a t u s " a r g u -
i m p o r t a n t c o n c e p t u a l p r o b l e m s and s e r i o u s p r a c t i c a l ments t h a t a l l o w q u e r i e s t o b e made a b o u t w h e t h e r
challenges. For e x a m p l e , i t would b e i n f e a s i b l e o r n o t t h e f u n c t i o n has been c o m p l e t e d . *
f o r a s i n g l e p r o b l e m - s o l v i n g system ( u s i n g a s i n g l e
model) t o attempt t o c a l c u l a t e the long chains o f Once b e g u n , t h e e x e c u t i o n o f any f u n c t i o n
p r i m i t i v e a c t i o n s needed t o p e r f o r m l e n g t h y t a s k s . proceeds e i t h e r u n t i l i t I s completed i n i t s
A way a r o u n d t h i s d i f f i c u l t y i s t o p r o g r a m a number n o r m a l manner o r u n t i l i t i s h a l t e d b y one o f a
o f c o o r d i n a t i n g " a c t i o n - u n i t s , " e a c h w i t h i t s own number o f " a b n o r m a l " c i r c u m s t a n c e s , s u c h a s t h e
p r o b l e m - s o l v i n g s y s t e m and m o d e l , and e a c h r e s p o n - v e h i c l e bumping i n t o u n e x p e c t e d o b j e c t s , o v e r l o a d
s i b l e f o r p l a n n i n g and e x e c u t i n g a s p e c i a l i z e d c o n d i t i o n s , r e s o u r c e e x h a u s t i o n , and s o o n . Under
function. I n p l a n n i n g how t o p e r f o r m i t s p a r t i c u - o r d i n a r y o p e r a t i o n , i f e x e c u t i o n o f MOVE r e s u l t s
l a r f u n c t i o n , e a c h a c t i o n - u n i t knows t h e e f f e c t s o f i n a bump, m o l l o n i s s t o p p e d a u t o m a t i c a l l y b y a
executing f u n c t i o n s handled by various of the other s p e c i a l mechanism o n t h e v e h i c l e . T h i s mechanism
action-units. W i t h t h i s k n o w l e d g e i t composes i t s can be o v e r r i d d e n by a s p e c i a l i n s t r u c t i o n from
p l a n a s a sequence o f o t h e r f u n c t i o n s ( w i t h t h e t h e c o m p u t e r , h o w e v e r , t o e n a b l e t h e r o b o t t o push
a p p r o p r i a t e a r g u m e n t s ) and l e a v e s t h e p l a n n i n g r e - ob J e c t s .
q u i r e d f o r each o f t h e s e f u n c t i o n s u p t o t h e
a c t i o n - u n i t s r e s p o n s i b l e f o r e x e c u t i n g them a t t h e The p r o b l e m - s o l v i n g s y s t e m s f o r MOVE and TURN
time they are to be executed. a r e t r i v i a l ; t h e y need o n l y t o c a l c u l a t e what s i g -
n a l s s h a l l b e sent t o r e g i s t e r s a s s o c i a t e d w i t h
Such a s y s t e m o f i n t e r d e p e n d e n t a c t i o n - u n i t s t h e m o t o r s i n o r d e r t o c o m p l e t e t h e d e s i r e d number
i m p l i e s c e r t a i n a d d i t i o n a l p r o b l e m s i n v o l v i n g com- of steps.
m u n i c a t i o n o f i n f o r m a t i o n and t r a n s f e r o f c o n t r o l
between u n i t s . When such a s y s t e m i s I m p l e m e n t e d At a l e v e l j u s t above MOVE and TURN is a f u n c -
o n a s e r i a l c o m p u t e r w i t h l i m i t e d c o r e memory, o b - t i o n whose e x e c u t i o n c a u s e s t h e v e h i c l e t o t r a v e l
v i o u s p r a c t i c a l d i f f i c u l t i e s a r i s e connected w i t h d i r e c t l y to a point s p e c i f i e d by a p a i r of ( x , y )
s w a p p i n g p r o g r a m segments i n and o u t o f c o r e and coordinates. T h i s f u n c t i o n i s implemented i n the
handling interrupts in real time. The c o o r d i n a t e d FORTRAN r o u t i n e LEG. The model used by LEG c o n -
a c t i o n - u n i t scheme s e r v e s a s a u s e f u l g u i d e i n e x - t a i n s i n f o r m a t i o n about the r o b o t ' s p r e s e n t ( x , y )
p l a i n i n g the o p e r a t i o n o f o u r system, even though l o c a t i o n and h e a d i n g r e l a t i v e t o a g i v e n c o o r d i n a t e
p r a c t i c a l n e c e s s i t i e s have d i c t a t e d o c c a s i o n a l d e - s y s t e m and i n f o r m a t i o n a b o u t how f a r t h e v e h i c l e
v i a t i o n s f r o m t h i s scheme i n o u r i m p l e m e n t a t i o n . t r a v e l s f o r each step a p p l i e d t o the s t e p p i n g
In the next s e c t i o n we s h a l l d i s c u s s the problem- motors. This information is stored along with
s o l v i n g p r o c e s s e s and m o d e l s a s s o c i a t e d w i t h some some o t h e r s p e c i a l c o n s t a n t s i n a s t r u c t u r e c a l l e d
s p e c i f i c f u n c t i o n s o f t h e p r e s e n t SRI r o b o t s y s t e m . t h e PARAMETER MODEL. Thus f o r a g i v e n ( x , y )

* Our i m p l e m e n t a t i o n a l l o w s a p r o g r a m c a l l i n g
r o u t i n e s l i k e MOVE o r TURN t o r u n i n p a r a l l e l
w i t h the motor f u n c t i o n s they i n i t i a t e .

-510-
d e s t i n a t i o n as an argument of LEG, LEG's problem- Special "model maintenance" programs insure
s o l v i n g system c a l c u l a t e s appropriate arguments t h a t the GRID MODEL is a u t o m a t i c a l l y updated by
f o r a TURN and MOVE sequence and then executes i n f o r m a t i o n about empty and f u l l f l o o r areas
t h i s sequence. Predicted changes in the r o b o t ' s gained by e i t h e r successful execution or i n t e r -
l o c a t i o n and heading caused by execution of MOVE r u p t i o n of MOVE commands.
and TURN are used to update the PARAMETER MODEL.
The PL program f i r s t uses the GRID MODEL to
Ascending one more l e v e l in our system, we e n - compute a network or graph of "nodes." The nodes
counter a group of FORTRAN " t w o - l e t t e r " r o u t i n e s correspond to p o i n t s in the room opposite corners
whose execution can be i n i t i a t e d from the t e l e - of o b s t a c l e s ; the shortest path to a goal p o i n t
t y p e . Our a c t i o n - u n i t system ceases to be s t r i c t - w i l l then pass through a sequence of a subset of
ly h i e r a r c h i c a l at t h i s p o i n t , since some of the these nodes. In F i g . 2 we show a complete GRID
t w o - l e t t e r commands can cause others to be exe- MODEL of a room c o n t a i n i n g three o b j e c t s . The
cuted . r o b o t ' s p o s i t i o n , marked " R , " and the goal p o s i -
t i o n , marked " G , " together w i t h the nodes A,B,C,D,
One of these t w o - l e t t e r commands, EX, takes as E , F , H , I , J and K are shown o v e r l a i d on the GRID
an argument a sequence of ( x , y ) coordinate p o s i - MODEL. The program PL then determines that the
t i o n s . Execution of EX causes the robot to t r a v e l shortest path is the sequence of p o i n t s , R , F , I ,
from i t s present p o s i t i o n d i r e c t l y t o the f i r s t and G by employing an o p t i m a l graph-searching a l g o -
p o i n t in the sequence, thence d i r e c t l y to the r i t h m developed by H a r t , et a l . 5
second, and so on u n t i l the robot reaches the l a s t
point in the sequence. The problem-solving system If the GRID MODEL map of the world contains
f o r EX simply needs to know the e f f e c t caused by unknown space, PL must decide whether or not to
execution of a LEG program and composes a chain of t r e a t t h i s unknown space as f u l l or empty. Cur-
LEG r o u t i n e s , each w i t h arguments provided by the r e n t l y , PL m u l t i p l i e s the length of any segment of
successive p o i n t s s p e c i f i e d in the sequence of the route through unknown space by a parameter k.
points. Under o r d i n a r y o p e r a t i o n , if one of these Thus if k = l , unknown space is t r e a t e d as empty;
LEG r o u t i n e s is h a l t e d due to a bump, EX backs the values of k g r e a t e r than u n i t y cause routes through
v e h i c l e up s l i g h t l y and then h a l t s . A s p e c i a l known empty space to be p r e f e r r e d to p o s s i b l y
f e a t u r e o f our implementation i s the a b i l i t y t o s h o r t e r routes through unknown space.
arm and s e r v i c e i n t e r r u p t s (such as caused by
bumps) at the FORTRAN programming l e v e l . Execution of TE is accomplished by f i r s t r e a d -
ing and processing a p i c t u r e (using PI w i t h the
Another t w o - l e t t e r command, P I , causes a p i c - camera aimed at the goal p o s i t i o n ) and t a k i n g a
t u r e to be read a f t e r the TV camera has been aimed r a n g e - f i n d e r r e a d i n g . The i n f o r m a t i o n about f u l l
at a s p e c i f i e d p o s i t i o n on the f l o o r . The problem- and empty f l o o r areas thus gained is added to the
s o l v i n g system f o r PI thus c a l c u l a t e s the appro- GRID MODEL. A route based on the updated GRID
p r i a t e arguments f o r a TURN r o u t i n e and a head- MODEL is then planned using PL, and then EX is exe-
t i l t i n g r o u t i n e ; PI then causes these to be exe- cuted using the arguments c a l c u l a t e d by PL. If the
c u t e d , reads in a p i c t u r e from the TV camera, and EX c a l l e d by TE is h a l t e d by a bump, a procedure
performs processing necessary to e x t r a c t informa- attempts to manuever around the i n t e r f e r i n g ob-
t i o n about empty areas on the f l o o r . ( D e t a i l s of s t a c l e , and then TE is c a l l e d to s t a r t over a g a i n .
the p i c t u r e processing programs of the robot sys- Thus, v i s i o n is used o n l y at the beginning of a
tem are described in Sec. I l l below.) journey and when unexpected bumps occur along the
journey.
The a b i l i t y to t r a v e l by the shortest route to
a s p e c i f i e d goal p o s i t i o n along a path c a l c u l a t e d Although our present robot system does not
to avoid bumping i n t o obstacles is provided by the have manipulators w i t h which to pick up o b j e c t s ,
t w o - l e t t e r command TE. Execution of TE i n v o l v e s it can move o b j e c t s by pushing them. The funda-
the c a l c u l a t i o n of an a p p r o p r i a t e sequence of mental a b i l i t y to push o b j e c t s from one place to
p o i n t s f o r EX and the execution of EX. This appro- another is programmed i n t o another t w o - l e t t e r
p r i a t e sequence is c a l c u l a t e d by a s p e c i a l problem- FORTRAN r o u t i n e , c a l l e d PU. Execution of PU causes
s o l v i n g system embodied in the t w o - l e t t e r command the robot to push an o b j e c t from one named p o s i t i o n
PL. along a s t r a i g h t l i n e path to another named p o s i -
t i o n . The program PU takes f i v e arguments: the
The source of i n f o r m a t i o n about the world used ( x , y ) coordinates of the o b j e c t to be pushed, the
by PL is a planar map of the room c a l l e d the GRID " s i z e " or maximum extent of the o b j e c t about i t s
MODEL. The GRID MODEL is a h i e r a r c h i c a l l y o r g a - center of g r a v i t y , and the ( x , y ) coordinates of the
nized system o f f o u r - b y - f o u r g r i d c e l l s . I n i t i a l l y spot to which the o b j e c t is to be pushed. The
the "whole w o r l d " is represented by a f o u r - b y - f o u r problem-solving system f o r PU assembles an EX, a
array of c e l l s . A given c e l l can be e i t h e r empty TURN, and two MOVE commands i n t o a sequence whose
(of o b s t a c l e s ) , f u l l , p a r t i a l l y f u l l , o r unknown. execution w i l l accomplish the d e s i r e d push. F i r s t
Each p a r t i a l l y f u l l c e l l i s f u r t h e r subdivided i n t o a l o c a t i o n from which the robot must begin pushing
a f o u r - b y - f o u r a r r a y o f c e l l s , and s o on, u n t i l a l l the o b j e c t is computed. Then PL is used to p l a n a
p a r t i a l l y f u l l c e l l s represent areas o f some s u i t - route to t h i s goal l o c a t i o n . The sequence of
ably s m a l l s i z e . (Our present system s p l i t s c e l l s p o i n t s along the route serves as the argument f o r
down to a depth of three l e v e l s , r e p r e s e n t i n g a EX t h a t is then executed. (Should EX be stopped by
smallest area of about 12 I n c h e s . ) a bump, PU is s t a r t e d over a g a i n . ) N e x t , PU's

-511-
p r o b l e m - s o l v i n g system (using the PARAMETER model) s i t u a t i o n , PUSH ( x , p , B ) , w i l l r e s u l t i n which the
c a l c u l a t e s an argument f o r TURN t h a t w i l l p o i n t o b j e c t x w i l l be at p o s i t i o n p. The second axiom
the robot i n the d i r e c t i o n t h a t the o b j e c t i s t o Btates t h a t any o b j e c t w i l l stay i n i t s o l d place
be pushed. A l a r g e argument is provided f o r the in the new s i t u a t i o n r e s u l t i n g by pushing a d i f -
f i r s t MOVE command so t h a t when it is executed, it ferent object.
w i l l bump i n t o the o b j e c t to be pushed and a u t o -
m a t i c a l l y h a l t . A f t e r the bump and h a l t , the In a d d i t i o n to the two axioms Just mentioned,
automatic stopping mechanism on the v e h i c l e is we would have o t h e r s d e s c r i b i n g the present p o s i -
o v e r r i d d e n and the next MOVE command is executed tions of o b j e c t s . For example, i f OBI i s a t c o -
w i t h an argument c a l c u l a t e d to push the o b j e c t the o r d i n a t e p o s i t i o n ( 3 , 5 ) i n the present s i t u a t i o n ,
desired distance. we would have:

B. Higher Level Functions POSITION (OBI, ( 3 , 5 ) , PRESENT) .

As we ascend to h i g h e r l e v e l f u n c t i o n s , the (This i n f o r m a t i o n is provided a u t o m a t i c a l l y by


r e q u i r e d p r o b l e m - s o l v i n g processes must be more r o u t i n e s t h a t scan the GRID MODEL, g i v i n g names to
powerful and g e n e r a l . We want our robot system to c l u s t e r s o f f u l l c e l l s and n o t i n g the l o c a t i o n s o f
have the a b i l i t y t o perform tasks p o s s i b l y r e q u i r - these c l u s t e r s . )
i n g q u i t e complex l o g i c a l d e d u c t i o n s . What is
needed f o r t h i s type of p r o b l e m - s o l v i n g is a gen- I n proving the t r u t h o f the c o n j e c t u r e , the
e r a l language in which to s t a t e problems and a theorem-prover used by QA-3 also produces the
powerful search s t r a t e g y w i t h which t o f i n d s o l u - place p and s i t u a t i o n s t h a t e x i s t . That i s , QA-3
t i o n s . We have chosen the language of f i r s t - o r d e r determines t h a t the d e s i r e d s i t u a t i o n s i s :
p r e d i c a t e c a l c u l u s i n which t o s t a t e h i g h l e v e l
problems f o r the r o b o t . These problems are then s = PUSH (0B3, ( 3 , 5 ) , PUSH (0B2, ( 3 , 5 ) , PRESENT)).
solved by an a d a p t a t i o n of a "Question Answering
System" QA-3, based on " r e s o l u t i o n " theorem- A l l of the i n f o r m a t i o n about the world used by QA-3
proving methods.6-9 i n s o l v i n g t h i s problem i s stored i n the form o f
axioms in a s t r u c t u r e c a l l e d the AXIOM MODEL. In
As an example of a h i g h l e v e l problem f o r the g e n e r a l , the AXIOM MODEL w i l l c o n t a i n a large num-
r o b o t , consider the t a s k of moving (by pushing) ber of f a c t s , more than are necessary f o r any
t h r e e o b j e c t s to a common p l a c e . This t a s k is an given d e d u c t i o n .
example of one t h a t has been executed by our p r e s -
ent system. If the o b j e c t s to be pushed a r e , say, Another LISP program examines the composition
OBI, 062, and 0B3, then the problem of moving them of f u n c t i o n s c a l c u l a t e d by QA-3 and determines
to a common place can be s t a t e d as a " c o n j e c t u r e " those lower l e v e l FORTRAN t w o - l e t t e r commands
f o r QA-3: needed to accomplish each of them. In our present
example, a sequence of PU commands would be assem-
C7p,s){POSITION ( 0 B l , p , s ) bled. I n order t o c a l c u l a t e the a p p r o p r i a t e a r g u -
A POSITION (OB2,p,s) ments f o r each PU, QA-3 is c a l l e d a g a i n , t h i s time
A POSITION ( 0 B 3 , p , s ) } to prove c o n j e c t u r e s of the form:

(That i s , "There e x i s t s a s i t u a t i o n s and a (^p,w){POSITION (OB2,p,PRESENT) A SIZE (OB2,w)) .


place p, such t h a t OBI, 0B2, and 0B3 are a l l at
place p in s i t u a t i o n s . " ) The task f o r QA-3 is to Again the proof produces the p and w t h a t e x i s t ,
" p r o v e " t h a t t h i s c o n j e c t u r e f o l l o w s from "axioms" thus p r o v i d i n g the necessary p o s i t i o n and size
t h a t describe the present p o s i t i o n of o b j e c t s and arguments f o r PU. (Size i n f o r m a t i o n is a l s o a u t o -
the e f f e c t s o f c e r t a i n a c t i o n s . m a t i c a l l y entered i n t o the AXIOM MODEL by r o u t i n e s
t h a t scan the GRID MODEL.)
Our f o r m u l a t i o n of these problems f o r the
theorem-prover i n v o l v e s s p e c i f y i n g the e f f e c t s o f In t r a n s f e r r i n g c o n t r o l between LISP and
a c t i o n s in terms of f u n c t i o n s t h a t map s i t u a t i o n s FORTRAN (and a l s o between separate large FORTRAN
i n t o new s i t u a t i o n s . For example, the f u n c t i o n segments), use Is made of a s p e c i a l m i n i a t u r e moni-
PUSH ( x , p , s ) maps the s i t u a t i o n s i n t o the s i t u a - t o r system c a l l e d the VALET. The VALET handles
t i o n r e s u l t i n g by pushing o b j e c t x i n t o place p. the process of d i s m i s s i n g program segments and
Thus two axioms needed by QA-3 to solve the push- s t a r t i n g up new ones using a u x i l i a r y drum storage
i n g problem a r e : f o r t r a n s f e r r i n g i n f o r m a t i o n between programs.

(Vx,p,s){POSITION ( x , p , PUSH ( x , p , s ) ) } The QA-3 theorem-proving system allows us to


pose q u i t e general problems to the robot system,
and but f u r t h e r research is needed on adapting theorem-
p r o v i n g techniques t o robot p r o b l e m - s o l v i n g i n
(Vx,y,p,q,s){POSITION ( x , p , s ) A ~ SAME ( x , y ) order t o Increase e f f i c i e n c y . * The g e n e r a l i t y o f
*=> POSITION (x,p,PUSH ( y , q , s ) ) } .

The f i r s t o f these axioms s t a t e s t h a t i f i n a n * We can e a s i l y propose l e s s f o r t u n a t e a x l o m a t i z a -


a r b i t r a r y s i t u a t i o n s, an a r b i t r a r y object x is t l o n s f o r the " c o l l e c t i n g o b j e c t s t a s k " t h a t
pushed to an a r b i t r a r y place p, then a new would prevent QA-3 from being able to solve i t .

-512-
theorem-proving techniques tempts us to use a The f i r s t o f the l o c a l operations i s a d i g i t a l
s i n g l e theorem-prover (and axiom s e t ) as a problem- d i f f e r e n t i a t i o n used to f i n d p o i n t s where there are
s o l v e r (and model) f o r a l l h i g h l e v e l robot a b i l i - s i g n i f i c a n t changes i n l i g h t i n t e n s i t y . These
t i e s . We might conclude, however, t h a t e f f i c i e n t changes u s u a l l y occur at or near the boundaries of
o p e r a t i o n r e q u i r e s a number of c o o r d i n a t i n g a c t i o n - o b j e c t s , as can be seen from F i g . 3 b . The next
u n i t s t r u c t u r e s , each having i t s own s p e c i a l i z e d step i s t o determine the l o c a l d i r e c t i o n o f these
theorem-prover and axiom set and each responsible boundaries. This is done by s y s t e m a t i c a l l y p l a c -
f o r r e l a t i v e l y narrow classes o f f u n c t i o n s . i n g small masks over the d i f f e r e n t i a t e d p i c t u r e ,
and l o o k i n g f o r places where the masks l i n e up w e l l
Another LISP program enables commands s t a t e d w i t h the g r a d i e n t . F i g . 3c shows the l o c a t i o n s and
in simple E n g l i s h to be e x e c u t e d . 1 0 , * 1 1 It also o r i e n t a t i o n s of masks t h a t responded s t r o n g l y .
accepts simple E n g l i s h statements about the e n -
vironment and t r a n s l a t e s them i n t o predicate c a l - The next step is to f i t these short l i n e seg-
culus statements to be stored as axioms. E n g l i s h ments w i t h longer s t r a i g h t l i n e s . This is done by
commands are o r d i n a r i l y t r a n s l a t e d i n t o p r e d i c a t e f i r s t grouping the short l i n e segments, and then
c a l c u l u s c o n j e c t u r e s f o r QA-3 to solve by produc- f i t t i n g a s i n g l e s t r a i g h t l i n e t o a l l o f the seg-
ing an a p p r o p r i a t e sequence of subordinate f u n c - ments in a group. Grouping is a systematic p r o -
t i o n s . For some simple commands, the theorem- cedure in which short segments are l i n k e d if they
prover is bypassed and lower l e v e l r o u t i n e s such are s u f f i c i e n t l y close i n l o c a t i o n and o r i e n t a t i o n .
as PU, TE, e t c . , are c a l l e d d i r e c t l y . F i g . 3d shows the r e s u l t s of f i t t i n g longer l i n e s
to the segments in each group. The f i n a l step is
The E n g l i s h program also accepts simple to J o i n the endpoints of these long l i n e s to p r o -
E n g l i s h questions t h a t r e q u i r e no robot a c t i o n s . duce a connected l i n e d r a w i n g . This is done by
For t h e s e , it uses QA-3 to discover the answer, c o n s i d e r i n g the endpoints one at a time and c r e a t -
and then i t d e l i v e r s t h i s answer i n E n g l i s h v i a i n g candidate c o n n e c t i o n s — s t r a i g h t connections to
the t e l e t y p e w r i t e r . (Task execution can also be neighboring e n d p o i n t s , e x t r a p o l a t i o n s t o p o i n t s o f
reported by an a p p r o p r i a t e English o u t p u t . ) intersection, extrapolations to T-junctions, etc.
The candidate that best f i t s the corresponding p a r t
Ill VISUAL PERCEPTION of the d e r i v a t i v e p i c t u r e is the one s e l e c t e d . The
f i n a l l i n e drawing produced by t h i s procedure is
V i s i o n is p o t e n t i a l l y the most e f f e c t i v e means shown i n F i g . 3 e .
f o r the robot system to o b t a i n i n f o r m a t i o n about
i t s w o r l d . The robot l i v e s i n a r a t h e r a n t i s e p t i c While the l i n e drawing preserves much of the
but nevertheless r e a l world of simple o b j e c t s — i n f o r m a t i o n in the quantized p i c t u r e in a compact
boxes, wedges, w a l l s , doorways, e t c . I t s v i s u a l form, i t o f t e n contains flaws due t o missing o r
system e x t r a c t s i n f o r m a t i o n about t h a t world from e x t r a l i n e segments t h a t complicate i t s a n a l y s i s .
a c o n v e n t i o n a l TV p i c t u r e . A complete scene a n a l y - C u r r e n t l y , the only i n f o r m a t i o n we e x t r a c t from
s i s would produce a d e s c r i p t i o n of the v i s u a l scene, the l i n e drawing is a map of the open f l o o r space.
i n c l u d i n g the l o c a t i o n and i d e n t i f i c a t i o n o f a l l A program c a l l e d f l o o r boundary f i r s t f i n d s a path
v i s i b l e o b j e c t s . C u r r e n t l y , we have two separate along those l i n e segments t h a t bound the f l o o r
o p e r a t i n g v i s i o n programs. One of these produces space in the p i c t u r e . These l i n e s are t y p i c a l l y
l i n e drawings, and has been used f o r some time to the places where the w a l l s or o b j e c t s meet the f l o o r ,
i d e n t i f y empty f l o o r space, regions on the f l o o r or where sides of o b j e c t s obscure our view of the
i n t o which the robot is f r e e to move. The o t h e r , f l o o r . F i g . 3f shows the f l o o r boundary e x t r a c t e d
which is more r e c e n t , locates and i d e n t i f i e s the from the l i n e drawing.
major, non-overlapping o b j e c t s . In t h i s section
we s h a l l give b r i e f d e s c r i p t i o n s of how these p r o - Now corresponding to any p o i n t in the p i c t u r e
grams o p e r a t e . is a ray going from the lens center of the camera
through the p i c t u r e p o i n t and out i n t o space. This
A. Line Drawing Program ray is the locus of a l l p o i n t s t h a t can produce the
given p i c t u r e p o i n t . If we f o l l o w the rays going
The l i n e drawing program produces a l i n e draw- through p o i n t s on the f l o o r boundary to the p o i n t s
i n g r e p r e s e n t a t i o n of a scene by a s e r i e s of essen- at which they pierce the f l o o r , we o b t a i n an i r -
t i a l l y l o c a l o p e r a t i o n s on a TV p i c t u r e . * F i g . 3a r e g u l a r polygon on the f l o o r t h a t bounds space
shows a t y p i c a l d i g i t i z e d p i c t u r e , which is stored known to be empty. In t h i s way the l i n e drawing
in the computer as a 120 x 120 array of 4 - b i t (16- is used to i d e n t i f y empty f l o o r space, and the
l e v e l ) i n t e n s i t y v a l u e s . The scene shown is f a i r l y v i s i o n system enters i n f o r m a t i o n about open area
t y p i c a l , and i n c l u d e s some of the problems of m i l d i n t o the GRID MODEL.
shadows and r e f l e c t i o n s , some f a i n t edges, and ob-
j e c t s not completely i n the f i e l d o f view. B. Object I d e n t i f i c a t i o n Program

Were the l i n e drawing program able to produce


a p e r f e c t l i n e drawing, the a n a l y s i s needed to
locate and i d e n t i f y o b j e c t s in the scene would be
* Most of these o p e r a t i o n s were adaptations of r e l a t i v e l y s t r a i g h t f o r w a r d . However, the l i n e
e a r l i e r work by R o b e r t s . 1 2 D e t a i l s of our p r o - drawing o f t e n contains flaws t h a t s e r i o u s l y com-
cedures, t o g e t h e r w i t h a d e s c r i p t i o n of s p e c i a l p l i c a t e i t s a n a l y s i s . Some of these flaws could
hardware f o r doing them e f f i c i e n t l y , are given be c o r r e c t e d by more elaborate l o c a l p r o c e s s i n g .
in R e f s . 1 and 4.

-513-
However, t h e r e is a l i m i t to how w e l l l o c a l p r o - scene o f F i g . 3a. The f i r s t o p e r a t i o n c a l l e d f o r i s
cessing can p e r f o r m , and when s i g n i f i c a n t edges a search f o r v e r t i c a l l i n e s , since these are u s u a l l y
cannot be t o l d from i n s i g n i f i c a n t edges on the both r e l i a b l y d e t e c t a b l e and s i g n i f i c a n t . F i g . 4a
basis of l o c a l c r i t e r i a , the goal of producing a shows t h a t the a p p r o p r i a t e operator found three v e r -
p e r f e c t l i n e drawing in t h i s way must be abandoned. t i c a l l i n e s , which happened to rank in confidence as
numbered.
The o b j e c t i d e n t i f i c a t i o n program l o c a t e s and
i d e n t i f i e s non-overlapping o b j e c t s b y g a t h e r i n g S t a r t i n g w i t h the highest confidence l i n e and
and i n t e r p r e t i n g evidence s u p p l i e d by l o c a l opera- checking to see t h a t i t s lower endpoint was w i t h i n
t o r s . The program c o n s i s t s of two p a r t s : a r e p e r - the p i c t u r e , the program next looked f o r o t h e r
t o i r e of l o c a l operators and an e x e c u t i v e . The l i n e s l e a v i n g t h a t e n d p o i n t . A f a i l u r e here would
l o c a l o p e r a t o r s process the g r a d i e n t p i c t u r e t o have led to the c o n c l u s i o n t h a t something was
perform t e s t s such as d e c i d i n g whether or not there s t r a n g e , and t h e r e f o r e to a t r a n s f e r to the next
i s a l i n e between two p o i n t s , o r f i n d i n g a l l l i n e s most c o n f i d e n t node in the t r e e . However, a " s p u r "
l e a v i n g a given p o i n t . Each operator r e t u r n s not was d e t e c t e d , as shown in F i g . 4b. Hypotheses t h a t
a s i n g l e answer, but a set of p o s s i b l e answers w i t h the lower endpoint was connected to the lower end-
associated confidences ( i d e a l l y , p r o b a b i l i t i e s ) p o i n t s of o t h e r v e r t i c a l s were r e j e c t e d because the
t h a t each answer is in f a c t c o r r e c t . The executive d i r e c t i o n of the spur was not c o r r e c t . Thus, at
explores the scene by c a l l i n g l o c a l o p e r a t o r s and t h i s p o i n t a t t e n t i o n was s h i f t e d to the top of the
e v a l u a t i n g the r e s u l t s i n the l i g h t o f both p r i o r v e r t i c a l . A spur was found t h e r e , as expected, and
t e s t r e s u l t s and b u i l t - i n knowledge o f the w o r l d . t h a t spur was f o l l o w e d to i t s endpoint as shown in
F i g . 4c. The f a c t t h a t i t s endpoint was on the
The e x e c u t i v e program is organized as a d e c i - p i c t u r e , coupled w i t h the f a c t t h a t the program had
s i o n t r e e . Each node in the t r e e s p e c i f i e s t h a t a f a i l e d i n i t s attempt t o connect the v e r t i c a l t o
p a r t i c u l a r t e s t is to be performed. The branches other v e r t i c a l s , provided s t r o n g evidence t h a t a
l e a v i n g a node correspond to the p o s s i b l e t e s t o u t - wedge had been found. A f u r t h e r check of the angle
comes, and since each outcome has an associated at the top of the v e r t i c a l confirmed the wedge hy-
c o n f i d e n c e , these confidences are attached to the p o t h e s i s , and the same c a l c u l a t i o n s used in the
branches. f l o o r boundary program were used to l o c a t e the lower
vertices.
A g i v e n node in the t r e e can be thought of as
r e p r e s e n t i n g a hypothesis about the contents of the At t h i s point one o b j e c t had been found and
scene. This hypothesis is simply t h a t the scene is i d e n t i f i e d , and a search f o r other o b j e c t s began.
p a r t i a l l y described b y the t e s t r e s u l t s s p e c i f i e d On the second i t e r a t i o n , the remaining v e r t i c a l s
by the path from the s t a r t node to the given node. were s u c c e s s f u l l y Joined at t h e i r lower e n d p o i n t s ,
The hypothesis is given a confidence by combining as shown in F i g . 4d, and v a r i o u s spurs were found,
the confidences of these t e s t r e s u l t s . The t e s t as shown in F i g . 4e. A s i m i l a r attempt to spot a
c a l l e d f o r at the given node is designed to provide wedge f a i l e d to produce s t r o n g evidence, as shown
a n answer t h a t w i l l tend t o c o n f i r m o r i n f i r m the i n F i g . 4 f , and the f i n a l output i n d i c a t e d the ex-
hypothesis. i s t e n c e and l o c a t i o n of an o b j e c t p a r t l y out of
view w i t h o u t s p e c i f y i n g what it was. At t h i s p o i n t
An a n a l y s i s of a scene proceeds as a search of there were no more v e r t i c a l l i n e s , and the a n a l y s i s
the d e c i s i o n t r e e d e s c r i b e d . At any stage in the was completed.
search we have a p a r t i a l l y expanded t r e e c o r r e s -
ponding to the t e s t s already performed. The nodes The o b j e c t i d e n t i f i c a t i o n program is capable of
a t the t i p s o f t h i s p a r t i a l t r e e , which w e s h a l l l o c a t i n g and o f t e n i d e n t i f y i n g non-overlapping ob-
c a l l open nodes, present us w i t h a choice of pos- j e c t s on the b a s i s of p a r t i a l i n f o r m a t i o n . There
s i b l e next t e s t s ( o r , a l t e r n a t i v e l y , hypotheses t o are a number of obvious ways in which it can be
be f u r t h e r i n v e s t i g a t e d ) . The open node w i t h h i g h - improved and extended, and f u r t h e r research w i l l be
est associated confidence i s selected f o r expansion, devoted to these t a s k s . However, even as it stands
i . e . , the t e s t c a l l e d f o r b y t h a t p a r t i c u l a r node it can provide the robot w i t h much v a l u a b l e i n f o r -
is performed. The search proceeds u n t i l the open mation about the r o b o t ' s w o r l d .
node w i t h h i g h e s t confidence is a t e r m i n a l node of
the t r e e . A t e r m i n a l node t y p i c a l l y represents a IV CONCLUSIONS
complete d e s c r i p t i o n of at l e a s t a p o r t i o n of the
scene, and hence c o n s t i t u t e s at l e a s t a p a r t i a l There are s e v e r a l key q u e s t i o n s t h a t our work
"answer," ( C e r t a i n t e r m i n a l nodes correspond to has helped to put i n t o f o c u s . Given t h a t a robot
impossible p h y s i c a l s i t u a t i o n s ; i n t h i s e v e n t , the system w i l l i n v o l v e the s u c c e s s f u l i n t e g r a t i o n o f
search Is resumed at the next most c o n f i d e n t node.) p r o b l e m - s o l v i n g , m o d e l l i n g , and p e r c e p t u a l a b i l i t i e s ,
A f t e r r e t u r n i n g a p a r t i a l answer the p o r t i o n of the there are many research questions concerning each
scene c o n t a i n i n g the o b j e c t found is d e l e t e d and of t h e s e . Let us discuss each in t u r n .
the a n a l y s i s begins a g a i n . Thus the t r e e search is
i t e r a t e d u n t i l the scene c o n t a i n s n o f u r t h e r o b - A. Problem-Solving
jects.
Our somewhat h i e r a r c h i c a l o r g a n i z a t i o n of
The d e c i s i o n t r e e i t s e l f embodies the s t r a t e g y problem-solvers and models seems a n a t u r a l , even if
f o r searching the scene. The basic ideas behind ad hoc, s o l u t i o n to o r g a n i z i n g complex b e h a v i o r .
t h i s s t r a t e g y are s i m p l e , and w i l l be i l l u s t r a t e d Are t h e r e a l t e r n a t i v e s ? W i l l the use of theorem-
by f o l l o w i n g the o p e r a t i o n of the program on the p r o v i n g techniques provide enough g e n e r a l i t y to

-514-
permit a s i n g l e general-purpose problem-solver, or The m a i n theme o f t h e p r o j e c t has b e e n , and
w i l l s e v e r a l " s p e c i a l i s t " t h e o r e m - p r o v e r s b e needed w i l l c o n t i n u e t o b e , t h e problem o f system i n t e g r a -
to gain the required e f f i c i e n c y ? tion. I n s t u d y i n g r o b o t systems t h a t i n t e r a c t w i t h
t h e r e a l w o r l d , i t seems e x t r e m e l y i m p o r t a n t t o
O t h e r q u e s t i o n s c o n c e r n t h e use o f t h e o r e m - b u i l d and p r o g r a m a r e a l s y s t e m and t o p r o v i d e i t
p r o v i n g methods f o r p r o b l e m - s o l v i n g . How d o t h e y w i t h a real environment. Whereas much can b e
compare w i t h t h e " p r o d u c t i o n m e t h o d s " a s used b y learned by s i m u l a t i n g c e r t a i n of the necessary func-
t h e G e n e r a l P r o b l e m S o l v e r (GPS) o r w i t h t h e p r o - t i o n s (we use t h i s s t r a t e g y r e g u l a r l y ) , many i m -
c e d u r a l language approach as developed by F i k e s ? 1 3 p o r t a n t issues are l i k e l y not to be a n t i c i p a t e d at
P e r h a p s some c o m b i n a t i o n o f a l l o f t h e s e w i l l p r o v e a l l in simulations. Thus q u e s t i o n s r e g a r d i n g , s a y ,
s u p e r i o r t o any o f t h e m ; p e r h a p s more e x p e r i e n c e the f e a s i b i l i t y o f a system o f i n t e r a c t i n g a c t i o n -
w i l l show t h a t t h e y a r e o n l y s u p e r f i c i a l l y d i f - u n i t s f o r c o n t r o l l i n g a r e a l r o b o t can o n l y b e c o n -
ferent . fronted by a c t u a l l y attempting to c o n t r o l a r e a l
r o b o t w i t h such a s y s t e m . Questions r e g a r d i n g the
Another question i s : T o what l e v e l o f d e t a i l s u i t a b i l i t y o f c a n d i d a t e v i s u a l p r o c e s s i n g schemes
s h o u l d b e h a v i o r a l p l a n s b e made b e f o r e p a r t o f t h e c a n most r e a l i s t i c a l l y b e a n s w e r e d b y e x p e r i m e n t s
p l a n i s e x e c u t e d and t h e r e s u l t s c h e c k e d a g a i n s t w i t h a s y s t e m t h a t needs t o " s e e " t h e r e a l w o r l d .
perceptual information? Although t h i s question T h e o r e m - p r o v i n g t e c h n i q u e s seem a d e q u a t e f o r s o l v -
w i l l n o t have a s i n g l e a n s w e r , w e need t o know i n g many " t o y " p r o b l e m s ; w i l l t h e f u l l g e n e r a l i t y
upon what f a c t o r s t h e answer d e p e n d s . o f t h i s approach r e a l l y b e e x p l o i t a b l e f o r d i r e c t -
i n g the automatic c o n t r o l o f mechanical equipment
Our p r o b l e m - s o l v i n g r e s e a r c h w i l l a l s o b e in real-time?
d i r e c t e d a t methods f o r o r g a n i z i n g even more com-
plex robot behavior. W e hope e v e n t u a l l y t o b e The q u e s t i o n s t h a t w e have posed i n t h i s s e c -
a b l e t o d e s i g n r o b o t systems capable o f p e r f o r m i n g t i o n a r e among t h o s e t h a t must b e answered i n o r d e r
complex assembly t a s k s r e q u i r i n g the i n t e l l i g e n t t o d e v e l o p u s e f u l and v e r s a t i l e r o b o t s y s t e m s .
use o f t o o l s and o t h e r m a t e r i a l s . E x p e r i m e n t i n g w i t h a f a c i l i t y such a s w e have d e -
s c r i b e d a p p e a r s t o b e t h e b e s t way t o e l i c i t t h e
B. Modelling p r o p e r q u e s t i o n s and t o w o r k t o w a r d t h e i r a n s w e r s .

S e v e r a l q u e s t i o n s a b o u t m o d e l s can b e p o s e d :
REFERENCES
Even i f w e c o n t i n u e t o use a number o f p r o b l e m -
s o l v e r s , must e a c h have i t s own model? T o what
1. N. Nilsson, et a l , "Application of Intelligent
e x t e n t c a n t h e same model s e r v e s e v e r a l p r o b l e m -
Automata to Reconnaissance," C o n t r a c t AF30(602)-
solvers? When a p e r c e p t u a l s y s t e m d i s c o v e r s new
4 1 4 7 , SRI P r o j e c t 5 9 5 3 , S t a n f o r d R e s e a r c h
i n f o r m a t i o n about the w o r l d , should it be entered
I n s t i t u t e , Menlo P a r k , C a l i f o r n i a ( f o u r I n t e r i m
d i r e c t l y i n t o a l l models c o n c e r n e d ? I n what f o r m
R e p o r t s and one F i n a l R e p o r t d a t e d December
s h o u l d i n f o r m a t i o n b e s t o r e d i n t h e v a r i o u s models?
1968).
S h o u l d p r o v i s i o n s b e made f o r f o r g e t t i n g o l d i n -
formation? Can a r o b o t s y s t e m be g i v e n a s i m p l e
2. C . A . Rosen and N . J . N i l s s o n , " A n I n t e l l i g e n t
m o d e l o f i t B own p r o b l e m - s o l v i n g a b i l i t i e s ? En-
A u t o m a t o n , " IEEE I n t e r n a t i o n a l C o n v e n t i o n
s u i n g r e s e a r c h and e x p e r i e n c e w i t h o u r p r e s e n t
Record, Part 9 (1967).
system should h e l p us w i t h these q u e s t i o n s .

3. B. Raphael, "Programming a R o b o t , " P r o c . IFIP


C. Visual Perception Congress 6 8 , E d i n b u r g h , S c o t l a n d (August 1968).

The i m m e d i a t e v i s i o n p r o b l e m s i n v o l v e i n c l u d -
4. G . E . F o r s e n , " P r o c e s s i n g V i s u a l Data w i t h a n
i n g more t e s t s i n t h e o b j e c t i d e n t i f i c a t i o n p r o g r a m Automaton E y e , " i n P i c t o r i a l P a t t e r n R e c o g n i -
t o c o m p l e t e u n f i n i s h e d a n a l y s i s , and r e m o v i n g t h e t i o n (Thompson Book Company, W a s h i n g t o n , D . C . ,
r e s t r i c t i o n to non-overlapping objects. Beyond 1968).
t h e s e improvements t h e r e are s t i l l l o n g e r range
problems to be s o l v e d . The scene a n a l y s i s p r o g r a m s 5. P . H a r t , N . N i l s s o n , and B . R a p h a e l , " A F o r m a l
i m p l i c i t l y s t o r e i n f o r m a t i o n about the w o r l d i n Basis f o r the H e u r i s t i c Determination of M i n i -
their structure. Changes i n t h e r o b o t ' s w o r l d c a n mum C o s t P a t h s , " IEEE T r a n s , on Systems S c i e n c e
r e q u i r e e x t e n s i v e changes t o t h e whole p r o g r a m . and C y b e r n e t i c s , V o l . SSC-4, N o . 2 , p p . 1 0 0 -
What p r o g r a m o r g a n i z a t i o n w o u l d m i n i m i z e t h e s e 107, ( J u l y 1968).
problems? How c a n t h e scene a n a l y s i s p r o g r a m i n -
t e r r o g a t e and use f a c t s s t o r e d i n t h e m o d e l t o 6. C . Green and B . R a p h a e l , "The Use o f T h e o r e m -
advantage? Since " f a c t s " obtained from e i t h e r the Proving Techniques i n Question-Answering
model o r t h e s u b r o u t i n e s are s u b j e c t t o e r r o r , i t S y s t e m s , " P r o c . 1968 ACM C o n f e r e n c e , Las V e g a s ,
i s n a t u r a l t o accompany them b y a c o n f i d e n c e mea- Nevada ( A u g u s t 1 9 6 8 ) .
sure. How s h o u l d t h e s e c o n f i d e n c e s b e computed
and how s h o u l d t h e y b e c o m b i n e d , s i n c e , l o o s e l y 7. B. Raphael, "Research on I n t e l l i g e n t Question-
s p e a k i n g , we o p e r a t e under c o n d i t i o n s of s t r o n g Answering Systems," F i n a l Report, Contract AF
s t a t i s t i c a l dependence? How c a n w e augment t h e c u r - 1 9 ( 6 2 8 ) - 5 9 1 9 , SRI P r o j e c t 6001, Stanford
r e n t r e p e r t o i r e o f s u b r o u t i n e s w i t h o t h e r s t o make R e s e a r c h I n s t i t u t e , Menlo Park, C a l i f o r n i a
use o f s u c h p r o p e r t i e s a s c o l o r , t e x t u r e and r a n g e ? (May 1 9 6 8 ) .
Future v i s i o n research w i l l be devoted to answering
q u e s t i o n s such as t h e s e .
-515-
8. C. Green, "Theorem-Proving by R e s o l u t i o n as a
Basis f o r Question-Answering Systems," Machine
I n t e l l i g e n c e 4, B. M e l t z e r and D. M i c h i e , Eds.
(Edinburgh U n i v e r s i t y Press, Edinburgh, S c o t -
l a n d ; to appear 1969).

9. C. Green, " A p p l i c a t i o n s of Theorem-Proving to


P r o b l e m - S o l v i n g , " P r o c . o f the I n t e r n a t i o n a l
J o i n t Conference o n A r t i f i c i a l I n t e l l i g e n c e ,
Washington, D.C., (May 1969).

10. L. S. Coles, "An On-Line Question-Answering


System w i t h N a t u r a l Language and P i c t o r i a l
I n p u t , " Proc. 1968 ACM Conference, LAS Vegas,
Nevada (August 1968).

11. L. S. Coles, " T a l k i n g w i t h a Robot in E n g l i s h , "


Proc. o f the I n t e r n a t i o n a l J o i n t Conference
o n A r t i f i c i a l I n t e l l i g e n c e , Washington, D.C.,
(May 1969).

12. L. G. R o b e r t s , "Machine Perception of Three-


Dimensional S o l i d s , " O p t i c a l and E l e c t r o -
O p t i c a l I n f o r m a t i o n Processing (MIT Press,
Cambridge, Massachusetts, 1965).

13. R. F i k e s , "A 9tudy in H e u r i s t i c Problem-


S o l v i n g : Problems Stated as P r o c e d u r e , "
Proc. of F o u r t h Systems Symposium, held at
Case Western Reserve U n i v e r s i t y , C l e v e l a n d ,
Ohio, November 1968 ( t o be p u b l i s h e d ) .

-516-
FIG. 2 A GRID MODEL OF A ROOM WITH THREE OBJECTS

-518-