AD—A 045 Web OHIO STATE UNIV COLUMBUS COMPUTER AND INFORMATION Sc—ETC F/S 9/2

A NATUR AL LANGUAGE GRAPHICS SYSTEM.(u)

JUN 77 0 C BROWN, S c KWASNY AFOSR—72—2351
UNCLASSIFILO OSU— CISRC— TR—77—8 A FOSR—TR—77—1229 NI.

. _ _

_ _

t6nas I k
~~~~ ~~
i

I I _
~A91 fl 1 U U
—
l f END
_ _

_ _ _ _ _ _
12-77
 ~ 28 ~ 25

::
:~ ~
~~~

,
, f~I~2.o
~ •
~

mu I .25 L•4 IIW

I.o

N
-. N’ f ’ i ‘hi
 TECHNICAL REPORT SERI ES

~~~~~~~~

DDC

E~UMPUTEE 1
dI~1fUPJ1iRT~UI~1
SE1~E~1~!E
1 ESER
~ E1 1TEEI

THE OHIO STATE UNIVERSITY COLUMBUS,OHIO

 ~~~~~~~~~~~~~~
- -

OSU-CISRC—TR— 77~~

A NA T URAL LANGUAGE GRAPHICS SYSTEM

BY

David C . Brown
Stan C. Kwasny

Computer and Information Science Research Center

The Ohio State University
Columbus , Ohio 43210
June , 1977

________ J
~ ~~~)
I
PREFACE

The Computer and Information Science Research Center of The Ohio State
Universi ty is an interdiscip linary research organ ization consisting of staff ,
graduate students , and facul ty of many University departments and laboratories.
This repor t describes research undertaken in cooperation with the Department
of Computer and Information Science.
Dur ing the summer of 1976 when this research was performed , the two
authors were supported by the T.E. French Fellowship and the Department of
Computer and Information Science. These two sources also provided computing
funds and technical support.
Par ti al suppor t by Grant AFOSR 72—2351 , B. Chandrasekaran , Princ ipal
Inves tigator , is gra t e f u l l y acknowledged .

•
—

~2Ct ~Ofl
13
0

~~~ ~~~~
~~

H ii
 ACKNOWLEDGEMENTS

• The authors would like to acknowled ge the help and support of the
following :

a. Dr. B. Chandrasekaran and Dr. N. K. Sondheimer for their constant

encouragement, valuable ideas, and constructive criticism during this
work;
b. Dr. H. W. Buttelmann and Dr. A. P. Lucido for their contributions during
the initial discussions of the project;
c. Dr. S. C. Shapiro for allowing us to use a version of the MENTAL pack-
age;
d. Nick Eastridge for help ing us with a number of details for the LISP—
FORTRAN interface for the DEC System 10, and for providing us with some
software; and
e. Dave Rypk.a for assistanc e with the programming of the LISP—FORTRAN
interface , and as a source of Plasma Panel folk—lore.

iii •

•
—~~ —--—-•-——-••— -——— - —- .•— ••—•—— -— • -•-•- —--—& - .
 0. CONTENTS. PAGE

1. INTRODUCTION •

1.1 Overview
1.2 Examp le o f Dialogue
1.3 Motivation
1.4 Repor t Outline

2. OVERALL SYSTEM DESIGN 3

2.1 General Descr iption
2.2 Sys tem Cons truction and Start—up
2.3 The Exec utiv e Module
2.4 The Error Module
2.5 The Initialization Module
2.6 Ty p i c a l Co nt r o l Flow

3. THE LANGUAGE ANAL YSIS MODULE 8

3.1 Function and Overall Design
3.2 The Two Inpu t Modules
3.3 Parsing
3.3.1 The Augmented Transition Network
Model
3.3.2 Lex ical Entries
3.3.3 The Grammar
3. 4 Repres entation of the Parse — the MC Interface
3.4.1 Valid Case Form s
3.4.2 Crea ting the Prototype
3.5 The QA In terface
3.6 Problem s in LA

4. THE LANGUAGE GENERATION MODULE 16

4.1 Func tion and Overall Design

5. THE KNOWLEDGE BASE MODULE 18

5.1 Func tion and Overall Design
5.2 The Seman tic Memory
5.2.1 Structure
5.2.2 Ac tIons
5.2 .3 Naming and Erasing

iv

~~~~~~ ~~~~~~~~~~~~~ •—--—— •~~~ - • — ~~~~— • - • • •-

• --—~~~~~~~~~~~ — . --•~~~~~~~~~~~~ •— --•- ~~~~~~ -~~~~~~-~~ • •
 1

5.3 Ques tion Answering

5.3.1 IntroductIon
5.3.2 The INFO Func tion
5.3.3 The FINDOBJ Func tion
5.3.4 The CONVERT Func tion
5.3.5 The FINDDRAW Function
5.4 Memory and Graphics
5.4.1 Introduction
5.4.2 Drawing
5.4.3 Erasing
5.4.4 Naming
5.5 Problem s and Extensions
5.5.1 Moving
5.5.2 Remembering
5.5.3 Defaul ts
5.5.4 Structural Informa tion
5.5.5 Defined Poin t s
5.5.6 Point Notation
5.5.7 Screen Model and Ac tion History
5.5.8 Func tions

6. THE GRAPHICS MODULE 31

6.1 Introduction
6.2 The Screen Func tions
6.3 Problems and Ex tensions

7. CONCLUSION 34
7.1 Sys tem Performance
7.1.1 General
7.1.2 Good Things
7.1.3 Bad Th ings
7.2 Ou tstanding Problem s
7.3 The Fu ture
7.4 Summary

8. REFERENCES 41

9. APPENDiCES 43

V
;.
—

I 1 ,~

I. INTRODUCTION
1.1 Overv iew

~~~~~ T~~~This repor t describes an experimental system for drawing simple

p ic tures on a computer graphics terminal using natural language input . The
system is capable of draw ing lines , points , and c ircles on c ommand from the
user , as well as answer ing q u es ti ons abou t sys tem capab il iti es and objec t s
on the screen. Erasures are also permitted . Language input can be em-
bell ished with touches to convey positional Information .
The system was designed and imp lemen ted by the authors during Summer 1976 ,
was wr itten in LISP 1.6 , runs in about 40K words on a DECSystem—1O
computer , and d isp lays p ic ture s on an ag 6O Plasma Panel.
The system was implemented to test out ideas on system organization ,
t o es t ablish the v iab ili t y of c omb in ing lang uage and grap h i c s , and to
experimen t with appropriate A.I . techniques.

1.2 Examp le o f d ialogue

*? PLEASE DRAW A VERTICAL 2 INCH LINE HERE CT> .
OK
*? PUT A POINT CALLED FRED HERE <T> .
OK
*? MAKE A CIRCLE WITH A TEN CM DIAMETER AT FRED .

OK
*? CONNECT FRED AND (100 ,150)
OK
*? CALL THE CIRCLE BALL

OK
*? ERASE THE LINE FROM FRED
OK
*? ERASE FRED
OK
*? CALL THE 2 INCH LINE BAT .
OK
*? DRAW A CIRCLE
OK
*? NAME THE CIRCLE FACE
OK
*? WHAT DID YOU DRAW HERE <T> ?
a LINE called BAT
*? WHAT CAN YOU DRAW?

LINES , POINTS , and CIRCLES

*? IS THERE A CIRCLE CALLED FACE?
yes
*? CAN YOU DRAW SQUARES?
no
The system prompt is “ * ?“ , and t he sys t em re sponse i s on t he f o l l o w i ng
line. The symbols “~ T~” indica te a touch on the screen . This extract
shows some of the variety of sentential forms available in the system. Not
sh own , are the results of the commands Ofl the screen .

1.3 Mo tivation
The research being do n e by o u r gr ou p pro mi ses t o be o n e more s t ep
towards the goal of natural interaction between man and computer. Our
work is based on the belief that use of more than one mode of communica—
tion is required to achieve that goal. We have chosen to investigate the
comb ina ti on of na tu ral language and graph ics. A system using this combi-
na tion allows the use of l ingu ist ic , graphical , or mixed forms for both
input and output of information . Careful development of this idea would
pr ov ide prac t ical sys t ems wi th a h ig h degree of habitability.
Many ar t is ts and desi gners wh o are ( and wish to rema in) n aive abo ut
programming will be able to interact productivel y with a natural language
graph ics system and such a system would allow a much wider group of people
to use computers. Special subsets of both language and pictures can be
developed for various uses. We are advocating a form of natural language
programming , but with an additional (graphical) mode of communication.
We feel that the feedback provided by the graphics will assist the user
in detecting, and then interactivel y correcting, err ors d u e t o vagu eness
or ambiguity. In addition , the user is able to select the appropriate
method for input of information and , f or example , could prov ide a r ough
draw ing of an object with a description of additional details. These
te c h n i q u e s would be v iable in a v a r i ety of appl ica tion s areas , such as
Anima t ion , Architecture , Eng ineering, and Educa tion .
Na tural Language Grap hics (NLG) provides a framework for research in
several areas of computer science. It supp lies a do m ain t~ r the stud y of
linguistic p henomena and l anguage unders t anding sys tem des ign. Memory and
inference will play a large role in an NLG system and consequently
knowled ge represen tation and manipulation is important. Such systems
would als o lead t o new ideas abo ut Comp ut er Crpah lcs , help ing to l ibera t e
i t f r om it s presen t al gori thmic approach. Man—machine interaction studies
and Al sys tems organization will also need to be pursued .
There has been no sys t ema ti c s t udy o f t he combina ti on of lang uage
and graph ics , al thoug h t her e have been a few sys t ems w it h t h i s comb ina tio n
[Klrsch , Coles , S imm ons , Badler and Win ograd ] and some recent work on
graphical information [Sondheirner , Palmer , Agin , Neva tia and Minsk y].
 -3

1.4 Report Outline

The next section , 2 , is co ncerned w it h t he overall d e si gn and o pera tio n
of the system. Sections 3 and 5 describe the language processing and know-
ledge co m ponen t s respe ct ivel y . As these are the longest and most important
components they are both described in some detail. The language
generation component is presented in Section 4, wh ile grap hical output is
d escribed in Section 6. The final section is a summary and con~ 1usion .

2. OVERALL SYSTEM DESIGN

2.1 General Description
NLC is composed of nine independent modules. Communication among
modules is achieved by a message—passing scheme in which any module may
invoke ano t her b y sending an appropriate message . As illustrated in Figure
SYS—1 , all messages are handed to the Executive module (EXEC) which
ultimately relays the message to its destination . Thus the system is
or ga ni zed he t erarch ic a ll y , a l t ho u gh a spe ci f ic c a l l i n g seq u ence has been
induced on the model which limits the interaction of modules in actual
operation.

LIN EXEC
H LFTN J

[
~
LA GR
~~ ~~~K B ] ~~~~~
J

FIGURE SYS— i
System Organi zat ion

_
 -- _ _

Each of the remaining eight modules has specific duties in the operation
of the system. The Initialization module (INIT) contains startup infor—
mat icn for NLG . If the system detects internal inconsistencies , the System
Err or module (ERR) is called to interact with the programmer in debugg ing.
Modules for graphical (touch) input (CI) and typed text input (IN)
provide preprocessed user input for language anal ysis. The Language
Analysis module (LA) parses the input , using an ATN gramma r , into a case— H
based semant ic structure. The Knowledge Base module (KB) develops a
semantic network from the input and creates instructions on how to update
the d isp lay screen. The Grap hics Output module (CR) uses these instruc—
tions to run a graphics program. Responses to the user are produced by
the Language Generation module (GEN) using a generative ATN grammar.
NLG is written in LISP 1.6 , except for the graphics primitive
routines which are coded in MA CRO—lO. As a debugg ing feat ure , the evaluat ion
of a LISP expression is permitted whenever the system prompts the user for
input by preced ing the expression with a dollar sign character ($).
Each module or collection of data used in building the system is
contained on a separate file. (A comp lete breakdown of the size of each
module appears in Appendix A.) In addition , a n as sembl y of utility
functions and functions useful in more than one module is kept on a
si ng le file. The ori ginal motivation for doing this was to facilitate the
dynam ic swapp ing of modules to maximize the use of memor y. Although the
mechanisms for this were implemented , processing t ime increased pr ohibitiv ely .
However , the comp lete separation of m odules in this fashion eliminated
many of the problems typ ically encountered in developing large systems .

2.2 System Construction and Startup

The system exists in LISP and MACRO—lO form. For ease of construction ,
a special control file is submitted to the monitor to build the system and
create a core—image “SAVE ” file. The content of the control file
follows :
.COM? LARITH ,ILSPF4 ,PLASUB
.R LISP 38;/A
1200 12400
*(SETQ ~~~~~~~~~~ T)
*(pUT5y~1 FLONUM FIXN UN )
*(LOAD NIL)
*LARITH ,ILSPF4 ,PLASUB $
*(DSKIN (NL C.L P2 . LSP))
* (DE CIMAL )
*(GC) . -

. SAV PLAS MA .SAV

.DEL PLASMA .HGH
.K/F
~~~-- - - - -- — -• - - - • _ _
 - -- --

~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~ ~~~~~

where $ represents an aitmode character.

First , the three MACRO— b files are comp iled : the file LARITH
establishes the interface between LISP and the FORTRAN library arithmetic
routines (i.e., SIN , COS , etc.); the file ILSPF4 builds the interface
from LISP to the graphics primitives package ; and the file PLASUB contains
the plasma panel subroutines. LISP then runs in 38K with the full word
space set at 1200 words (octal) and the free storage space s.~t a t 12400
word s (octal). These values were established mostly throug h exper imen-
tation , but see the LISP reference manual [Quam and Diffie] for further
d etails. Once in LISP , the KB trace mechan ism , used f o r deb u gg ing,
must be switched off by setting the flag *TRACEOFF* . The two internal
LISP routines FLONUM and FIXNUM , wh ich deal with converting f ixed a nd
floating numbers , must be put on the DDT symbol table to be available for
the interfaces. Next , the relocatable files compiled earlier are loaded
into expanded core. LISP makes use of the system loader which itself
is loaded on call and unloaded when finished. At this point , the total
core allocation for NLG may reach 43K. The system is loaded next. The
f ile NLGLP2.LSP contains the individual file names along with their
location on disk. Lastly, t he sys t em is placed in dec imal mode and g a r b a g e
collected. The comp le te system is now saved in the core—image file
“PLA SMA ” and the task is comp letad . The sketchy descr ip ti on g iven ab ove
may be c l ar i f ied s ing ificantl y be r e f e r r ing to the LISP manual ’s
sections on adjusting system parameters and loading b i n ary files.
Once im ~ig~ of NL G is ava il a b l e , a user types
RI! PLAS ~1A
to gain access to it. At the prompt , a val id messag e to INIT m ay be sent
v ia EXEC ; however , f or co nv enience , one m ay type
(INIT)
to establish contac t with NLG .

2.3 The Executive Module

The Executive modu le (EXEC) receives and sends all messages in the
sy stem. A validity test Is performed on messages according to the following
syn tax

The notation used in expressing syntax in this report is a modified

BNF where the symbol “or ” shows alternatives and subscri pts foll owing
non—terminals show the minimum number of occurrences.
~
!IUUuIUI.uIUuuuIUUIIIP1! ! — i ’-- -- - - ‘ —---- -‘.—. — - .. . ______ — . -
~ ~ ~~~ ~ ~~~~~~~ ~~ ~ ~~~ ~~~~~

,
~ ~~~

- I -s s ag e :: ( ~t o
- from . -~m a r k Irgs )
- to : := m o d u l e nam e
from :: m o d u l e name
~m .irk - ::= ~~~ or
- l rgs - ::= ( - irg )
The m o d u l e mimes used in t h e < t o ~- and < f r o m components must c o r r e s p o n d
to t h e names g i v e n in F i g u r e S Y S — l . The l i s t of ar g u m e n t s is not c h e c k e d
by EXEC , s i n c e each m o d u l e g e n e r a l l y protects itself f r o m bad d a t a .
M essages w i t h a q u e ry m ar k (;fl Q) a r e s e n t as m o d u l e c a l l s w h i l e thc , se
with .i r e s p ose m a r k (PR ) ar e r e t u r n e d as a v a l u e to t h e c a ll e r . Also ,
EX EC may he g i v en t h e m e s sa g e by c a l l i n g or r e t u r n i n g . T h is a l l o w s t h e
c o n v e y a n ce of m e s sag e s in f o u r ways , c o r r e s p o n d i n g to t h e m a n i p u l a t i o n of
the c a l l i n g leve l i n L I S P and t h e c a l l i n g leve l of NLC . The f o u r
possibilities are :
(i) c a b l i n g EXEC w i t h a i.Q message ;
(i i~ c a l l i n g EXEC w i t h a ~ R message ;
(ii i ) r e t u r n i n g a ~Q message t o EXEC ; and
(iv) r e t u r n i n g a ~ R message to EXEC.
The f i r s t c o m b i n a t i o n r e s u l t s in a c o n v e n t i o n a l m o d u l e c a b i w h i l e t h e l a s t
results in a c o n v e n t i o n al return. The second p o s s i b i l i ty is never used
in NL G . The t h i r d one a l l o w s m o d u l e i n v o c a t i o n in a manner s i m i l a r to
c o— r o ut in in g . LA a i d KB o c c a si o n a l l y c o m m u n i c a t e in t h i s f a s h i o n .
E XEC ’ s f i n a l d u t y is to a u t o m a t i c a l ly p r o d u c e a t r a c e of a l l s y s t e m
m e s s a ge s on t h e f i l e M E S S . L P T , i n c l u d i n g s t a t i s t i c s on t ime and space
u t i l iz at io n .

2. ~ The [r r r Module
The S y s t e m E r r o r m o d u l e ( E R R ) e x i s t s s o l e l y as a d e b u g g ing f e a t u r e .
If t h e message v a l i d i t y t e st c o n d u c t e d by EXE C f a i l s , t h e n ERR is c a l l e d .
I t d i s p l ay s t h e bad message an J i n t e r a c t s w i t h t h e u s e r to c r e a t e a new
one. Of c o u r s e , t h e d o l lar si gn ($) f e a t u r e w h i c h a l l o w s LISP e v a l u a t i o n
is supported allowing a k n o w l e d g e a b l e p e r s o n to comp l e t e l y investigate
the problem. ERR m -i v be c o n s i d e r e d an e x te n s i o n to EXEC , b u t has been
rar ely used .

2.5 The I n i t i a l i z a t i o n M o d u l e
The sy s t e m is a c t i v at e d w i t h a u s e r message to the mit i a l i z a t in n
module (IN I T) Here , a l l g loba l s y s t e m pa r a m e t e r s ar e in it L i i i zed , o u t p u t

—.
.

c h a n n e l s ar e e s t a b l i s h e d , a n d an m it ía ! screen erase is p e r f or m e d . The

s c r e e n ’ s c u r s o r p o s i t i o n is set to z e r o and t h e d i st a n ce b e t w e e n l i n e s

- - - - -- .- - - - - -
— - — - - -~~~~~ —~~~ - -- ~ --— .- -—
_ _ _
~~~~~~~~~
 - rr rr,, t~~~~.w ra ~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~

( of text on the screen is set to 15 screen units.

utilized in repositioning the cursor after drawing.
These parameters are
An introductory
system response is then made via CEN .
Next , a message is sent to activate LA for input processing. It
must be determined from the eventual reponse to this message whether
to stop the system or to re—activate LA. For the former , an exit
response is made and NLC halts , otherwise GEN is directed to respond
“OK ” and LA is called again.

2 .6 Typical Control Flow

Although NLG is based on a heterarchica l model , hierarchical relation—
ships exist among modules viewed in operation . The f o l i o w in g d i a g r am
-(-

illustrates typical control flow in the system

USER )

IN I T
~~ ~~~~
-

LA — KB GEN
~~ ~
/\
/
IN GEN GEN GR

GI

When INIT is activated by the user , it knows to expect input. Thus , LA

is called. From there , IN is activated to process the raw input , w ith
CI be ing called to handle graphical input. During LA ’ s processing, KB
may he consulted for stored or acquired knowledge . Upon completion , LA

Of course , EXEC controls the passing of messages , but the diagram w o u l d

become unnecessaril y complicated if EXEC were inc l uded everywhere. The
remainder of this paper will contain no mention of EXEC unles s essential
to the discussion.

~p .
~~~~~~~ tL~~~~ ~~~
-. _
.
~
 8

can either pass control to KB or return to INIT. The former happens for
commands and the latter for question—answering. When KB gets control ,
it concludes the processing by building memory structures and directing
CR to draw. KB then returns to INIT. GEN may be activated by either INIT ,
LA , or KB.

3. THE LANGUAGE ANALYSIS MODULE

3.1 Function and Overall Desi gn
The Language Analysis module (LA) processes all input to the system ,
including both typed text and touches to the screen. Figure LA—l shows
the log ical structure of LA.

IN I T

r - - - - - -
ir
1
1 _ _

Con~ oI
H --- - MG

- ~~~~~~~~~~~~

I ATN
IN Parse r ~ qacaI~~ 4~ QA

- . -~~ L
/1
•1 ‘
“

~
‘
~~~~~~~
I

GI Grammar ‘Lexicon I
I 1~ ./
/ ~~~
-~~~~~~ LA ~~~~~~~
H KB -

FIGURE LA—i
LA O r g a n i z a t i o n

An Au gmented Transition Network (ATN) parser [Woods , 1973] serves as

the primary component of the module , and a call of LA may optionall y
spec i fy an ATM state name or configuration . While parsing, t he gramm a r
permits queri e s to the Questi on—Answering (QA) mechanism within the

-- ,- - — .-~~~~~~~-— - -~~~~~- ~~~--.- ~~~~~~ - - - - - —- — - ~~~~~~ —~~ — m -~~ ~~~~~~~~~~~~~~~~~~

~ --

1 -1 9

Knowledge Base (KB), via the function QACALL , to retrieve other stored
knowledge as required. The parse results in a case—like structure called
a prototype which contains a representation of the ori ginal input (see
Section 3.4). If the input posed a question , the answer may then be
retr ieved from QA via the function QASK. For commands , the prototype
is passed as the argument in a message to the Memory and Grap hic s (MC )
mechan ism within KB. If the input commanded the system to stop, then a
return message is conveyed to INIT which stops the system.

3.2 The Two Input Modules

Before parsing can begin , the input must be scanned and assembled into
list form. The Input module (IN) is designed for this purpose. If IN
should detect a touch control character while scanning, then the Graphics
Input module (CI) is activated to interpret a touch. Touches become
screen coordinate pairs (a LISP dotted pair of numbers) in the input list.
Encountering a carriage—return causes IN to return the input list to LA
for parsing.

3.3 Pars ing

3 .3 .1 The Augmented Transition Network Model
The ATN parser used in this module is modeled after the one used
in the LUNAR system [Woods , 1973; Woods , et al. , 19721 . A finite state
network of states and arcs is endowed with recursive capabilities allowing
a sub—network traversal to be performed in the course of following one
arc . This recursive transition network is further augmented by the
capacity to manipulate “reg isters ” and perform function calls during
trav ersals. Such reg is te rs can con t a in fl a gs , words from the input
string, partial parse trees , case structures , or any expression permitted
by the host language . This allows the various constituents to be built ,
tested , al t ered , and steered into position within the represe itation
chosen . The behavior of the parser is determined by the grammar and lexicon
pr ovided. The syntax of gramma r specification in an ATN appears in
A ppendix B. The lexicon consists of a list of feature—value pairs for
each word sense.

3.3.2 Lexi ca l Entries

In NLG , new features were added to lexical entries throughout

2_ _ _ _ _ _
 10

t h e p r o j e c t as t h e y become justifiable after discovering additional

d i s t i n c t i o n s to be made w h i l e p r o c e s s i n g . Four f e a t u r e s a r e c o n s i d e r e d
b a s i c to the p a r s i n g scheme : t h e s y n t a c t i c c a t e g o r y of t h e word sense
(CT GY); t h e root f o r m of the e n t r y (ROOT) ; t h e number c a t e g o r y (SING or
PLUR) f o r nouns (NUM) ; and the t r a n s i t i v i t y of verbs (T RANS). A case
feature (CASE) was also used to suggest which case or cases the entry
mi ght fit. For the position case , an indicator (TAG) showed the type of
pos ition being described by taking the values ENDPOINT or LOCATOR . For
verb s l ike “connect ” which literall y means “draw line ” , the object to be
drawn (
i.e., a strai gh t line ) is specified under the feature OBJ . Words
such as “c i r c u m f e r e n c e” or “diameter ” w h i c h app ly o n l y to c i r c l e s have
t h e f e a t u r e %OBJECT w i t h a v a l u e of (CIRCLE) to i n d i c a t e t h i s d e p e n d e n c e .
S i m i l a r ly , “degrees ” app lies as a size measurement to ang les and is
indicated by a SIZEOF feature with the proper value . Adjectiva l relation-
ships are utilized in disambi guation to limit the number of poss ible
word senses to consider. For example , a line ca n be STRAIGHT or CURVED
and th e feature ADJ is used to specif y this fact. Determiners can be
definite or indefinite indicated by a DEF fea ture with either a T or
NiL value . Lastly, abbreviations are distinguished by the feature
ABBREV . The input vocabulary appears in Appendix C while a partial lexicon
is given in Appendix D.

3 .3.3 The Gramma r

The gramma r for parsing can be viewed as five distinct networks :
sen ten c e , to uch , noun phrase , prepositional phrase , and question . The
five network diagrams appear in Appendices E , F , C , H , and I.
The sentence level network controls the construction of imperative
prototypes. The initia l state (S*) contains arcs which decide whether a
command or a question form is present. Questions typ icall y beg in with
auxiliaries or question words of various types and these are detected by
the function QSTART , called as the test portion of the arc which looks for
q uestions . Notice that the grammar permits prepositional phr ases or
touches to the screen at the beginning of a sentence. These are
remembered using the HOLD mechanism until their proper place can he found
later in the string. The imperative form is parsed from the state S*IMP .
If the system is being commanded to stop, then the arc t o s t a t e S*STOP
builds the proper message to be conveyed to INTl and parsing stops.
The arc to VP*HEAD is crucial in the development of the Imper ative

_ _ _ _ _
_ _
- - — —~~~~~.-~~~~~~ —
- -~~~~-- —-
--~~~~- - —
~~~~~~~~~~ -
~

11

proto type . It finds the verb near the front of the sentence and estab-
lishes the prototype . At state VP*HEAD , the post—verbial constituents are
expected. In most cases , a noun phrase will follow , but other possibilities
remain. For verbs like “connec t ” , a virtual object (ST—LINE) is implied .
This will be followed by an indication of what two things are to be connected.
Once at the state VP* , any of a number of constituents can be handled .
In particular , a touch is treated directly in the grammar in a manner
similar to noun phrases and prepositional phr ases. When the sentence has
been c omple t el y p a r s e d , the resulti ng prototype structure is returned as
the final value.
The touch network allows the parsing of a touch to the screen which
has been p r e p r oc essed b y CI , or equivalently, a specification in the input
string of a pair of screen coordinates. Either of these may be optionally
pr eceded b y “here” , “there ” , “this ” , or “ that ” . After a touch point has
be en parsed , the LISP dotted pair of x—y coordinates is returned as the
resul t. An arc has been provided at the accepting state of the touch
netwo rk to recognize multiple touches to the same point on the screen .
Th i s became necess a ry a f t er such a phenomenon was observed in oper at in g
the system . Of course , onl y one pa ir of coordinates is returned in such
cases.
The noun phrase network describes the processing of entities used
in the system as noun phrases. In addition to the standard noun phrase
consisting of the head noun with nominal modifiers , this network also
handle s:
(1) names — e.g., “the point 1°’ or simply “F ” ;
(2) the preposition “of” e.g., ~‘an ang le of 32 degrees ” or
—

“a l ine of length two inches ” ;

(3) a t o u ch u sed as a comp lementizer — e.g, “the l ine ~T~ ” ;
and
(
4) calls via QACALL to establish the identity of a name
(FINDOBJ) or to change numeric scale (CONVERT).
Sufficient flexibility is p rovided in the network to process most reasonable
combinations of the lexical items known to the system. The reader is
referred to the NP* network in Appendix C for a more detailed exposition.
The prep ositional phrase network provides an additional flexibilit y
in processing. The preposition is found first. This can be either a
standard preposition (e.g., to , from , a t , through , etc.) or one that takes

~~~~~~~~~~~ --- -. --
. --, ~~ - -— ---- - ~~~~~~~
.
 12

a compound form (e.g., between). Next , either a noun phrase or a touch

A
can be found . Finall y, if a compound form is expected , as in “be tween
the point P and here <T> ” , then this is parsed . Note tha t by using the
HOLD mechanism and a VIR arc , the interpretation given is literally
“be tween the point P and between here <T> ” . When comp lete , the phrase
is returned and a partial protot ype , b uilt from the ph r a s e , is lifted to
t he c al
l ing level.
The f inal compo nent of the gramma r is the question network. This
has been developed to the point where several types of ques ti ons u seful
in NLG can be processed correctl y, bu t rema in s somewha t inc omple te as
a general question—answering grammar. Nevertheless , the following questions
res ul t in answers a s in d i c a ted:
(1) What can you draw?
LINES , POINTS , and CIRCLES
(2) Can you draw circles?
yes
(3) Can you draw a circle with a 3 inch circumference?
yes
(4) How many scre en units is an inch?
64
(5) Is there a point named P7
yes
(6) Is there a circle named Fred here <T>?
no
(7) Is there a strai ght c i rcl e ?
I don ’ t know
The question network permits a variety of syntactic forms as shown in
the diagram in A ppendix I , but answers canno t be generated for some of
the possible paths . This situation was allowed to develop intentionall y,
since the system is not primarily concerned with this capability. More
work in this area is antici pated.
The five n e tworks combi n e to f o rm a gramma r f o r NLG wh ic h performs
w el eno u gh to parse most sentences In less than a half second — a m ajor
factor in allowing real time response. Append ix .1 contains some sample
sentences with their timings.

3.4 Representation of the Parse — the MG Interface

The result of a completed parse is a structure called a protot ype .
Appendix K contains a description of one w h i c h shows the form permitted
by each of the slots in the case—like structure. The f u n c t i o n of each of
these is discussed h r E e f l v below.

_ _ _ _ _
.
~~ --- . -
_
.— ~~~~- - -- - — -.. --- . -..- - -~~~-~~~~~~~~~~ .~~~~~~~~~~~~~~ .- -.-~~~~~~~~~~~~~~~~~~~~~~~~
~

3.4.1 Val id Case Forms

The %TYPE case indicates what form of input was detected during
parsing. Two types are distinguished by the system , na me l y t he impera tive
(IMP) form and the question (Q) form. Only imperative forms result in
calls on MC in the c u rren t imp lemen tation. Question forms are handled
completely within LA.
Three actions are possible for the %ACTION case. These are drawing,
erasing, and naming. The verb of the imperative sentence is reduced to
one of these basic forms .
Only three objects are recognized by the system. Thus , the %OBJECT
case can be f illed b y either ST—LINE , CIRCLE , or POINT .
The %NAME case is included to contain the name of the drawn or
erased object. This name is limited in form to a LISP atom and is
specified by the user in the input string.
The or ientation of a line can be specified in the ZORIENT case.
This contains a numeric value in degrees measured counter—clockwise
from the horizontal.
Position on the screen can be indicated in the %POSIT case. Any
n umber of po ints (usually no more than two ) may be specified. Each point
specification consists of a tag for either ENDPOINT , LOCATOR , o r TOUCH ,
and the coordinates of the actual point. ENDPOINT is used to indicate
the end poin ts of a line. The midpoint of a l ine , the center of a circle ,
and th e location of a po int are all indicated by the tag LOCATOR. A
TOUCH tag is used whenever the function of the point is not determined from
the input sentence (e.g., “Era se this -:T~ ”). The coordinates can be
specified in actual screen units as a dotted pair of numbers , or they may
be g iven b y referring to the node identifier for a particular point
stored in the KB. The latter is indicated by the tag NODE dotted to the
node identifier.
F inall y, the %SIZE case indicates the length of a strai ght l ine or
the radius of a circle in screen units. In addition , by including one
o f t he ta gs RADIUS , DIAMETER , o r CIRCUIIFERENCE , t h e size of a circle can
be given in terms of one of these.

3.4.2 Crea ting the Prototype

The prototype is established direct lv from the verb . The f u n c t Ion
DEFINEPROTO is used to create a propert y list entr y for t hu verb
consisting of a list of the case names to appear In the finished p r ototyp e

S. . -
--- -.--- --———-—— - — — - - - --.-- - — — - -
-—-———- ,- ~~~~~~~~~~~~ - . - ~~ - -.—-- -- --- . .-
 . .- .
~ -

p
14

form. At present , prototypes for draw , erase , and name are determined
in this way. They consist of exactly the same form , althoug h this need
not necessarily be the case. Other verbs , of course , appear in the
input string, but these are reduced to the canonical verb form so that
the reference to the prototype is always done correctly. The function
PROTO is used in the ATN gramma r to retrieve the proper prototype form.
As constituents are discovered and parsed by the ATN gr amm ar ,
they are slotted into the proper case positions. The function PROTOTYPE
changes the current prototype to reflect the value to be added . Normall y,
additions are made as they are discovered , but the HOLD mechanism
allows some of these decisions to be delayed until a later VIR arc can
find the proper positionin g of the constituent. This is particularly
useful in handling touches which are found out of p lace and f o r mov ing
prepositional phrases to the post—verbial position.

3.5 The QA Interface

The Language A n al yzer utilizes knowled ge accumulated in KB to hel p
in parsing and to answer user questions. The two functions , QACALL and
QASK provide the interface to QA for grammar—produced and user—initiated
questions respective ly (s ee F igure LA—i)
QACALL build s a message from the arguments passed to it. The first
argument indicates the name of the operation required of QA and this is
l imited to either FINDOBJ , CONVERT , FINDDRAW , or INFO . Other arguments
t o QACALL are included as part of the message to QA. Only FINDOBJ and
CONVERT are used in grammar—produced questions , while user—initiated
questions may use FINDDRAW and INFO .
QASK is activated by the LA Controller after the parser returns with
an indication that it parsed a question form. This function must determine
h ow to answer a user ’s question based on reg ister settings created during
the pa rse. The collection of reg isters is passed as the function ’ s one
argument. If the user ’s answer has not alread y been determined during
the pa rse , then an appropriate message is sent to retrieve additional
information from QA.
The function FINDOBJ is used in determining the reference of a name
mentioned in the input string. For phrases like “ the point P” it answers
the quest ions “ Is the object named P a point ?“ and “Wha t is the node
id entifier in memory for the point named p ? ” The parser c a l l s on this
function whenever a name is mentioned . This is especially useful In

_ _ _ _ _
 15

determining how a word is being used when no lexical entry can be found .
Conversions to system units are accomplished through the CONVERT
feature of QA. The onl y units recognized i n MC a r e screen u ni t s for
leng ths and degrees for angles. Any other scale must be converted to
these and the gramma r contains appropriate calls to QA to accomp lish this.
FINDDRAW is used to determine if a particular entity is drawabl e
o r erasable , and to retrieve the list of objects drawable by the system .
It s use is lim it ed t o a n swer i ng spec i f ic q u es tio ns like “Can y ou draw ... “

or “What can you draw?”

The INFO feature of QA encompasses all other types of questions.
A prototype structure , similar to that built as a final form of
representation for completed sentences , is partiall y f i lled in w it h the
information known at the t ime of the call and passed as a parameter. In
addition , a l ist of the cases tha t need to be filled by INFO is also
passed. The response arrives in the form of a prototype with the requested
cases be ing filled i f possible. If more than one item can fill a case
slot , then all of these are returned . For example , in answering a question
like “What did you draw here <T>?” the question is put to INFO in the form
“Give me the %OBJECT and %NAME of the items drawn at <T> .” . The respon se
is lat er passed along to GEM so that the eventual answer comes out as ,
for example;
“a CIRCLE named BALL ”
“a POINT named FRED”

3.6 Problems in LA
A number of m inor problems continue to be a point of concern in the
lan g uage anal yzer. Of course , it is difficult to assess the ultimate limita-
t ions of the approach taken here , but hopefull y no maj or obs ta c l e s w ill
he encountered in the immediate future to impede the extendabi lity of the
system. A few problems are discussed below .
The adjectives “sm~’ll” and “l a r g e ” should be handled heuristically.
In doing this , the object must be considered and appropriate size cal-
culations must be made relative to its shape. The current approach is
adeq u a t e as a gu es s f o r av erage size ci r c les on t he scre en , hut a “one
inch d i ame ter ” small c i rcl e and a “four inch diameter ” lar ge c i rcle m ay
not always be appropriate. This problem is not unique to these two words.
C ons i der , for example , how to handle “n ear ” or other locativ e s such as
“above ” , “below ” , “b e h i n d ” , or “n e x t to ” .

— -~~ —- - --- _ - ---- — - ---- - — --. ---~~~~~~ - — — - --- — —-— ---- - - - ,--
 16

Prepositiona l phrases containing “ to ” and “from ” ar e h and l ed

identicall y. They each result in creating an ENDPOINT tag in t h e %POSIT
case of the prototype. Thus , the directionality of the statement is
lost. Such a distinction was deliberately not made since a pr oblem
can arise in the graphics routines if a line is drawn in one direction
and an erasure is attempted in the other. Some simp le solutions to the
“end po int ” problem exist and should be forthcoming.
In pa rs in g , the grammar should perform more consistency tests
with objects and their parameters. Currently, the gramma r allows input
sentences to specif y circles drawn at angles , lines drawn with circum-
f e r e n c e s , and poi nt s t o be draw n “fr om here to there ” .
The prototype itself may not be easily extended to handle , for
exa m p le , a “move” operation . One manner of specification in a move
co u ld ind ica te an o r i gin and a destination position. This mig ht require
introducing sub—cases into the %POSIT case. Other new operations may
req uire similar extensions.
Th e v e rb largel y determines what structure gets built during parsing.
There may be no easy way out of this , but to extend the system to handle
s ome simple elli pt ical forms like “A circle here <T> please ” one may want
a more flexible scheme for determining the prototype.

4. THE LANGUAGE GENERATION MODULE

4 .1 Fu nct ion and Overall Desig n
The generation of language responses to the user in the system is
p erf ormed b y the Language Generation module (CEN). A small
collection of temp lates serves as the basis for generation (see Appendix
L). A module can request that a response he generated by sp ec i f y ing
a template number (TO , Ti , T2 , ...) and an ordered list of substitutions to
be made for the asterisks which appear in the template.
Besides this straightforward method , there is another way that responses
can get generated in the system. Instead of indicating the template
number and substitution list as the two arguments in messages sent to GEN ,
a message may contain the indi cator CENANS 1 or CENANS followed by a
pr ototype . This ind icates that either one or many answers are required
from the prototype given. A generative ATN grammar processes these
requests by parsing the prototype and pr oducing a list of temp late
numbers and substitution lists , one for each r e s po n s e . The gramma r

,-
— - -- -.- - - - ~-
~~~~~~~~ ~~~~~~~~~~~
 17

at this stage of development is rather simp le (see Appendix M) but shows

promise for future work.
The t echn ique s used in t he CEN m od u le work adeq u a t el y for NLG as it
now exists. It is hoped that as improvements evolve in other modules of
the system , CEN w il
l prove to be expandable as well.

~~~~
--
-
~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ---— ~~~~~~ - -— -~~~~~~~~ -~~~ -
-- L
 1.
18

5. KNOWLEDGE BASE NODULE

~~~
5.1 Function and Overall Design
This module accepts a message passed to it from som e other module
and performs one of its two major tasks depending on a marker in the first
part of the argument slot of the message . Control is passed to the
Memory and Graphics section (an MG marker) or to the Question Answering
section (a QA marker). The components of the KB module are shown in
Figure KB— i. The MENTAL functions [Shapiro 1974] occupy 1.5K words , the
in iti al knowled ge occup ies 3.5K words , and the rest of the KB LISP code
is 7K words. The module has the following functions :
i. to answer questions about the stored knowledge posed by
other modules in the system ,
ii. to build , e r a s e , and change semantic memory structures ,
iii . to produce a message to be sent to the Graphics module
which will control the drawing of an object on the screen .

5.2 The Semantic Memory

5 .2 . 1 Structure
The s t y le of network structures used is heavil y in f l uenced by th ose
d iscussed in Brachman [1976]. They are mani pulated and b u i l t u s i n g a
sl ightly modified version of MENTAL [Shapiro 1974]. The network is used
to encode both the system ’s prior knowled ge about objects and details of
the objects introduced during the discourse. Attached to the descri ptions
of objects in the network are actions (ACTs) which are encoded in LISP
and used for checking s tru c tu re s , producing part of the output message ,
and prov iding information if it is not specified by the user. App end i x
N g ives a list of the actions and their functions , the Table below
give’- a list of the labels used on the Semantic Network links , wh i le
Appendix 0 describes the initial state of the network.

~~~~~~~~~~~~ --~~~-— - --_ -

~ - - C “- ----‘r-’- r,_ r - ~~~ .r?In -~ v~~ .rw,.. ..r’ - -‘:;—:=-r’---——-- ~~~~~~~~~~~~~~~~~~~~~~~~~ —

19

Lf ~ KB

f ~ C T~ ty

~
‘
- /
~~ —-

‘
:~.-
~~~~~~~~~~~~~~~

L_ ,~ ~~ _
\ KS

P~~C(,I~ ’~ M

XE)’ :
1
~J . f
~
-
~~tt I~ .

L Is $~ ~~ C S S C

t
~~~
LI SP ~tt c ~cki~ to r’~~
— - —~~~~.

_ .z._____
?~~-tAcLI ,~e.tt

.. Co, ~tr o I / .l cw

Figure KB - i

-. - — *— -- -. — - .- - , . - -- , -
-
 y 2 . .—. -t .. -
— a.7~ r-.r- -~ , -- - —--—-,—-. —
~ ~~~~~~~~~~~ ~~~~~~~ ~~~~~~~~~~

20

TABLE OF LABELS FOR SEMAN TI C ~ [I W O R K LINKS

LABEL SL\N hS FOR C0~ 1ENT

SUBSETOF subset of connects two sets
ELMNTOF Element of connects item to a set
NAN 1~ Has name for item , or set , ut
concept
TYPE Has type for every node (SET ,
CONCEPT , ITEM , DESCRIP-
TION , ASSERTION , SYSTEM)
EL-INTDESCR Element description connects set to descrip-
tion of a typical element
DATTR Description of attribute from concept node to
description
ROLE Plays role f r o m d e s c r i p t i o n node
RESTR Is r e s t r i c t e d to connects descri ption
t o set
ACTION to a c t i o n f r o m d e s c r i p t i o n node
to action name
ATTR Has attribute connects item to
a s s e r t ion
INSTNCOF Is an instance of connects item to concept
VAL Has value connects assertion to
i t e m or v a l u e
NUMBER Number of o c c u r r e n c e s connects description
to a number
STRUCTURAL to a structural check connects concept to
an a c t i o n
INSTANTIATES I n s ta n t i a t e s connects assertion
t0 description
OBJECT ~ to e r a s e d ob j e c t
.
IU simple screen model
FRASED

The backbone of the network is a h i e r a r c h y of

s e t s , i n c l u d i n g ~t r a i gh i t
lines , points , ang les , distances , defined p a in ts (i.e., locati ens on the
screen; no t r .ecessarily illuminated), and circles (Figure KB—2 ).
Consider t h e portion of the network which stores prior information
about circles (Figure KB—3).
Node 1 represents the set of all Circles. Node 2 r e p r e s e nt s t h e
de scr ipt Ion of a typ ical element of th a t set Nn~Ie 3 r e p r e s e n t s one of
. the
parts of the desc r ipt ion . The part shown here is that o f “h.oc~~t or ” ,

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
__
 .
- . .. : : : .r r..rc rnr.r ;rr: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ,‘ .—— . ,, ---—--..- - —
~~~ ~ ~ , — --.—
~~ ~~ ——

ME
~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~ S E

Figure KB-2

b~~~~~~iP -r , ,~
‘-
~~~~~~

S. a - 9
:>~ET ‘r ’ ~ (i ~ t~ ~~

I ~~~~~ — -
-
~

\~‘-~ p ,4tS
~~F ’c ~
—

Figure KB - 3

~~~~~ r~~M Sc . Er ” E ’ o r ) r L
~ ~ I

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Figure KB-5 ~~~~~~~

- -— - .-- .
_____________
—
.
‘— ———
, ——-—
—. -= ‘ ‘ — ~--—~~~ -
rn
_ __. — - ,- . -5—- —— — — -.
5”

22

whi. h is O-, -.o - ia t , d with t h e -,,- PO SIT case in the Input prototype , and
.

rest r i c t ~ th i n ~~s w h i c h p l ay t h e role of locator to members of the set of

de l j ed j )O juts
~~ Nod e -4 ) .
( The ACTs w i l l he explained later.
Fi gure KB—4 shows , is a result of a request to draw on the screen~
how i n f o r m a t i o n a b o u t an object is stored in the network attached to the
description of that type of object. Subparts of the object are also
connected in this fashion. Notic e the ‘assertions ” in the network , wh ich
include “ the assertion (node 2) that the locator of the item (node 1) is
a particular defined point (node 3)” .

5.2.2 Actions
There are three types of actions attached to the network: the
Structural action , the Default action , and the Todraw action.
The Structura ~ act is used to check pieces of network after they have
been b u i l t to the specification g iven in the input protot yp e. For example
t h e net b u i l t f o r the new c i r c l e in F i g u r e KB—4 is m i s s i nc s i z e i n f o r -
mation (i.e., Diameter , Radius , or Circumference). ACT— l9 (Sec Figure
KB—3) will detect this omission and will report that ti:. role of Radius
has not been instantiated. In general the structural art ‘knows ’ wha t
the minimum requirements are for drawing the o bj e c t , and w i l l return a

l i s t of t h o s e m i s s i n g . In t h e case of a c i r c l e , t h e requirements a r e
Locator and Radius.
The Default acts are activated if a non—null list is returned b y a
Structural act. In this case the a c t uscd is the one associated with
the node which has a Role link to Radius (i.e., ACT—22 ; see Figure KB—3).
The Default act will look to see whether there is a ny t h i n g in t h e n e t w o r k
which will allow the missing value to be cal cu lated (e.g., Radius calculated
from Diameter), and if not , w i l l provide a reasonable default value . When
it has a value , it builds an additiona l p iece of network and completes
the requirements of the Structur al act. A Default action is run for every
omission. In the implemented system only circles have default actions
The Locator defaults to a touch point s l i g h t l y to the right of the center
of th e screen , and the Rad ius defaults to 100 screen units (approximately 1.5
inches).

For s i m p l i c i t y , many imi ortant links are not shown in Fi gure KB—4 . Note
that this is t h e f o r m of t h e n e t w o r k pri or to defaul ts.
~

23
Draw a circle called Ball at (100, 200)

i? .I. LE S
~

1>#Iri-~

~“0

. . . .I
L O c A T ~~

I4T1 ,
~~
~ 2

EM .~~

F
:
~~~~~~~~~~~~~~~~~~~~~~~m

L •

-4

~
~~
>~
-‘- ,
IM ~~1 I ’

- •
•
• ~~
:

2CC .~
z
‘1

Figure KB-4

- —--—- - -- - -—- - .
.- - - -- . — - - --~~--
 24

Once the item is full y specified the Todraw actio,~ can be used .
IL takes the piece of network representing the item to be drawn and
produces from i t a p r o g r a m f o r m w h i c h w i l l do t h e j o b . For e x a mp l e , t he
c i r c l e , with its center at the point (100 , 200) , and its defaulted
radius of 100 units , will pr oduce (F4CRCL 100 200 100). For a strai ght
line the Todraw act is a little more complicated as it has a choice
which depends on which combination of endpoint , midpoint , ang le , and l ength
has been specified . (See A ppendix P).

5.2.3 Naming and Erasing

The user has the ability to ask for an object to be drawn , and name
it , both in the same sentence (e.g., dr aw a circle called Ball at (100 ,200)).
This results in a Name link from the item as in Figure KB—4 . If a name
is not specified then the system provides one by taking the di gits from
the MENTAL node identifier (e.g., N2345) and app ending them to the first two
le tters of the concept name (e.g., C12345). Every node generated by
MENTAL is automaticall y g iven a unique identifier.
When the user requests that an object be erased from the screen the
system leaves it in the network , but marks it as erased. This arts as a
simple screen model , and prevents the system from attempting to erase an
obj ect that is alread y erased. (See Fi gure KB-5) .

5.3 Question Answering

5.3.1 In troduction
This section of the Knowledge Base is activated when another module
needs some information. The syntax of the various QA call s is given in
Appendix Q. There are four available functions:
i. INFO — A g e n e r a l n e t w o r k s e a r c h i n g f u n c t i o n , and t h e
most powerfu l function of the four.
ii. FINDOBJ — Used for discovering whether a word which is
suspected of being the name of an object actually
names some object on the screen.
iii. CONVERT — This is a function for converting lengths to
screen units or angles into degrees.
iv. FINDDR AW — Used for finding out whether an object can be
drawn .

5.3.2 The INFO F u n c t i o n

A p r o t o t y p e f o r m and •i i 1st of k e y w o r d s a r e p a s s e d as t h e two

,- -
~~~~~~~~~~~~ -- .
~~~~~~~~~~ ~~~~~~~ — rn—— - —~~~~~~~~~~~ - - - - -
-
‘. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

25

p a r a m e t e r s of INFO . The p r o t o t y p e f o r m is used to s e a r c h f o r i t e m s

in t h e n e t w o r k . A l i s t of i t e m s m a t c h i n g t h e description (i.e., t h e
p r o t o t y p e is v i e w e d as a d e s c r i p t i o n ) is o b t a i n e d and t h o s e i t e m s a r e
f u r t h e r s e a r c h e d f o r t h e i n f o r m a t ion r e q u e s t e d by t h e l i s t of keywords.
For e x a m p l e :
( INFO (%OBJECT %NANE NODE %POSIT)
( (%POSIT . ( (LOCATOR ( 10 0 . 2 0 0 ) ) ) )
) )
could give :
( (%OBJECT . (CIRCLE POINT) )
(70NANE . (BALL FRED) )
(NODE . (N234 5 N3442) )
(%POSI T • ( ( 100.200) )

i . e ., a l i s t of d o t t e d p a i r s of t h e form (keyword . list). The answer

to the q u e s t i o n “Which o b j e c t s are at (lOO , 200)” t h e r e f o r e , is a c i r c l e
named Ball and a p o i n t c a l l e d FRED.
The INFO f u n c t i o n is capable of h a n d l i n g p o s i t i o n s s p e c i f i e d by
LOCATOR , ENDPOINT , and TOUCH. The lat ter will successfull y match both
LOCATOR and ENDPOINT during the search , but preference will be given to
LOCATOR. Consequently, if the new inputs “What did you draw here <touch ’?” ,
and the touch point corresponds to the endpoint of a line and the center
of a circle then the response will name just the circle. However , if the
touch corresponds to two LOCATORS (or two ENDPOINTS) then both will be
returned.
The function uses the ITEMFIND procedure which searches for all
objects which match a given prototype. This is done by considering e a c h
o~ th e cases in the prototype separately (e.g., %NAME , %SIZE), obtaining
lis ts of matched items , and then form ing the intersection of those list s .
The resulting list gives the ITEM nodes of all objects which possess all
of the required characteristics.
There a r e two e r r o r c o n d i t i o n s f o r t h e INFO f u n c t . i o n s . If there is

no o b j e c t t h a t c o r r e s p o n d s to t h e d e s c r i p t i o n t h e n t h e message “object
as s p e c i f i e d c a n n o t be f o u n d ” is o u t p u t to the user. In t h i s case , and
in the case where the object is found but there is no Inform ation corres-
ponding to the given keyword , the “ list ’ returned with the Keyword is
NIL , e. g., ( ( % P O S I T . NIL)).

-- -- --.-- - --- .- - .----------------- ‘---- --- - ---—-- —---- - - .— —--

——
~ - _ _ _ _ _ _ _

26

5.3.3 The FINDOBJ Function

There are two ways to use this function. The first is to specify
both a suspected name and an object type in the call , e.g. , (FINDOBJ BALL CIRCLE) .
This tests the network to see if there is an object of the specified type
w ith the specified name . If there is , then the node identifier of that
item is returned , e.g ., ((NODE . N2345) CIRCLE ) . If there is no
object of that type with that name then NIL is returned , e.g ., (NIL CIRCLE).
If , by accident , an invalid object name has been given in the call , then
NIL is re tu rned ins t ead , e.g. , (NIL NIL). The second type of call allows
the objec t type not to be specified in the call (i.e., NIL is used). This
works in the same way as descr ibed ab ove , except that on return it
inserts the object type of the item with the name given in the call ,
e.g. , (FINDOBJ BALL NIL) gives (( (NODE . N2345) CIRCLE )).

5.3.4 The CONVERT Function

The response from this function is the value obtained after converting
the given number into screen units , or degrees , depending on the unit
specified , e.g., (CONVERT 3 INCH) g ives (192). Note that the screen is
8 inches square , and 512 b y 512 screen units .

5.3.5 The FINDDR AW F u n c t i o n

There are two uses for t h i s f u n c t i o n . The first allows the question
“can you d r a w an X? ” t o be asked by using the c a l l (F INDD RA W X ) . The
res pon se will be e it her T o r NIL , f or ye s or no , respective ly. The second
use is to answer the question “what can you draw?” , e.g., (FINDDRAW NIL)
gives (ST—LINES POINTS CIRCLES). This function operates by searching the
network for concepts with a TODRAW role. It is assumed that as theY can
be drawn , then they can also be named or erased.

5.4 Memory and Graphics

5. !i. l Introduction
On recei pt of the message , the KB module examines the first item
of the argument list. If it is “MG ” the Mem ory and Gr ap hics section
takes control. The %TYPE value is examined next and control passes to the
a p p r o p r i a t e s e c t i o n d e p e n d I n g on its value. Note that onl y imper atives
(IMP )are imp lement ed In MG. Next the iACTION v a l u e is used t o switch
to t h e NAME , ERASE , or DRAW sections of KB.

— -- - - .- —. - - - — - - -.- ——-- .-. -- . - - - - - -. -- - -- ~~~~— ,- . ~~~~ -- _ _ _

 I
27

5.4.2 Drawing
A f t e r f i n d i n g t h e set and concept nodes for the object to be
drawn , an item node is inserted into the network for this new object ,
and links are formed between it and the set and concept nodes. If a
name is given (e.g., “Draw a circle called Ball” ) this is added to the item
node , otherwise a name is generated by the system as previously described.
The orientation , position , and size information is extracted in turn
from the prototype , and appropr iate ATTR , VAL and INSTANTIATES links are
set up. Note that the positional information may include a
(NODE . <node identifier> ) form , due to sentences such as “connect P
with (100 .200) where P is the name of a point , and the node identifier is
that of the item node for the point .
The Structura l act is used to check the structure , as p rev iou s l y
described , and if necessary , default values are inserted in the network.
Once the net for the item is properl y spe ci f ied , the TODRAW action is used
to produce a program form for inclusion in a message which is then sent
to the GR module. This message contains a modified form of the prototype
that was sent to the KB module (Appendix P). After the drawing is
completed , the screen model is adjusted , and a message is sent to the
INIT module to indicate successful comp le tion.

5.4 .3 Erasing
If the screen is to be erased , then the screen model is altered to
mark all items as erased , and a message is sent to the CR module to erase
the screen. On return from CR , the KB module itself sends a successful
completion message . If an object is to be erased , and if that object has
not been directl y spec i f ied , then the network is searched for all items
that match the prototype . If more than one is found the items are checked
and al l those which have alread y been erased by the user are deleted from
t h e list. From the items remainin g, one is chosen to be erased. If the
object has been specified then the form (NODE . < node identifier ) will
be used. This removes the necessity for a n y further searching, and th at
o b j e c t can be erased d i r e c t l y . The TODRA W act i s used to p r o d u c e a program

Note that the positional information may include TOUCH forms . These
will ma tch both ENDPOINT and LOCATOR during the search.

-. L~~~~~~~~
 28

form which describes the erasure . This is included in a message

whi ch is sent to CR. The onl y d ifference between a draw and erase
message for the same object is that tile %ACTION is marked as DRAW or
ERASE , respectively. Consequently the program form can be produced
by exactl y the same Tod raw action for both ope rations. The CR module
detects the difference in the prototype and acts accordingly. After the
erasing is completed , the screen model is adjusted to mark that item as
era sed , and a message is sent to the INIT module to indicate successful
c omp letion.

5.4.4 Naming
No te that “naming ” in this sy s t e m r e f e r s to t h e n a m i n g of o b j e c t s
drawn on the screen , and does not mean the naming of groups of ob~ -.
~cts
(e.g. , four straight lines being called a square). The first action is to
r emove t he (%NAM E . < name> ) pair from the nrototvp e . and then use the
new prototype to search for all items that match that descri ption.
Here , as b e f o re , t h e positional information may include TOUCH forms .
Those that already have u s e r g i v e n names a r e d i s c a r d e d , and one item is
p icked from the resulting list. That item node is deleted from the net
and a new node is inserted having tile same links , except for the name
link which points to the new name . Finall y a message is sent to the
INIT module to indica t e s u c c e s s f u l c o m p l e t i o n .

5.5 P r o b l e m s and E x t e n s i o n s
5.5.1 Moving
There are three types of oper ations of this type that we would
consider adding to the system.
i. Draw in the same p lace with no change after a prior erasure .
e.g. , “Draw Li. ” (a null move)
ii. Draw in a different p lace with no e r a s u r e of o l d object.
e.g. , “Copy the circle here <touch > .”
iii. Draw in a different place with erasure of the old object.
e.g., “Move Li to here <touch ; .”
It would be nice if the mechanism which handles these c o u l d also provide
changes in t h e values of other cases on request (e.g. , “Draw LI here hut
at 45 degrees and •in i n c h sma 11cr ”). Many of these deta i h s have been
worked out , but have not been imp lement ed . One reason for not imp lementin g
th is was our susp icion that In the worst cases the prot otvp.- mi ght need
sli ght revisions . To do t h e above o p er a t ions at all rt’qu I r,’s spec ial

--- ~~~~~~~~~~~~~~~~~ ~~~~~~~~~ -— ~~~~~~. .- -

 --

29

interpretation of the protot ype in order to be able to specif y the existing

ob j e c t b e i n g r e f e r r e d to , as w e l l as t h e new requirements.

5.5.2 Remembering
We c o ns idered imp lementing an action that would allow the user to
‘remember ’ the ob j e c t s tha t h e had drawn by hav ing a l l or par t of the
semantic net dumped to a file. A ‘re ca l l ’ function could be used to
read back information from specified files (i.e., those created by th e
remember func tion) and re—establish it as the semantic net. This pair
of act ions could be used to maintain a record of the state of an interaction ,
and to save complete or partially completed drawings between sessions.
Reme m bered p ic tu res could also be d isp layed , after appropriate preprocessing,
on other devices such as a plotter or a CRT . In addition , it may be
usef u l to hav e a ‘f orget ’ function which could be used to permanentl y
erase ne t s t ruc t ures represe nti ng a l l , or p a r t , of some i t e m or items .
This could be used if a m i s t a k e had been made , or if some object was
no longer requ ired. It could be argued that the erase action should
remove the item from the network. However , this would remove the possibilit y
of redraw ing a named object after it had been erased .

5.5.3 Defaults
A si
m ple ex t ensio n of t he defa u lt m e c hanism w ou ld be to m ark al
l
p ieces of the net which were buil t as a result of default acts. This
would allow the system to recognize the difference , and enable the system
to remind the user of exactly what he had specified for an object. In
addition , defa ults could be provii4ed for all of the primitives , whereas
at present , only circles have default actions. The only additions
required would be to slot the default and structural acts into the code for
KB , and create links from the appropriate nodes of the concepts to the
act names. The activation mechanism will work for all defaults in the same
way .
Structural acts return a list of those roles which need to be i n s t a n -
tiated before the Tod raw act can function correctly. It may he the case
that t h e D e f a u l t a c t i o n s s h o u l d he e x e c u t e d in a p r e f e r r e d o r d e r so as to
make maximum use of the information given and to minimize the n u m b e r of
defa ults.
For a better treatment of defaults , two ex tra C cchn I qucs won Id hay .
to he used . The present Implementat ion of defa~i lts is essentiall y context—
fre e , and merely uses b u i l t - I n values. There w i l l he situations when th l ~

:i.
~~~-
~ — - - — —~~- - -. -. - ~~~~~~~~~~ - — ~~~~~ —-- .— - - - — -— -- - - --- ~~ --—
~ -
- -——-- - — -
~~~~~~~~~ — --—-- ~~~~~ _ _

30

is not sufficient , and where a heuristic default mechanism would be

appropriate. For examp le , a d e f a u l t action could execute a space—finding
r o u t i n e to f i n d a r e a s o n a b l e l o c a t i o n f o r t h e object on the screen. In
other S i t u a t i o n s i t would be s e n s i b l e to ask t h e user f o r the e x t r a
i n f o r m a t i o n , if , f o r example , t h e h e u r i s t i c fails to find a suitable value.

5.5.4 Structural Information

There are strong ground s for arguing that the Structural act
contains too much information , even for such simple items as point , l ine ,
and circle. This information , for example what full y defines a straight
l ine , is not accessible as it is encoded in the LISP code. For even onl y
slightly more comp lex o b j e c ts , such as squar e , it appears that this infor-
mation should be encoded in a network form. This would allow a new use
of QA by LA. During parsing , to aid in establishing the role of prepositions ,
it would be useful to have LA ask QA what other cases to input , given the
current input . For example , having fo un d an ang le and endpoint , and some-
th ing else wh ich is no t ea sil y p arsed , QA would be able to suggest
try ing to look for a length description . A structura l network would pl ay
the important role of specif ying the relationships between the various
attribute descriptions.

5.5.5 Def ined Points

At present , whenever a defined point is specified , no attempt
is made to determine whether there is already a structure for that
point. A simp le extension of KB would be to include a routine to check
for existing structures. This would mean that lines which shiared the
same endpoint would share the same Def point structure.

5.5.6 Point Notation

It mi ght be p o s s i b l e t o use a unified notation for points , lines ,
and ar eas. For example:
a particular point . 100 , 200-’
t he line y = 200 - vx , 200’
a n y poin t , with y = 200 ~ ? , 2O O ~
any po in t < ? , ?>
the’ screen - vx
, vy-
a h o r i z o n t a l l ine segment < 50 — 70 , 200-
a rectangle - — 70, 100 — 150 -
any rectangle ~ xI — x2 , y l — v2~
In a more s o p h i s t i c a t e d s y s t e m it might he usefu l to have volumes as
primitives and other basic objec t s is degraded volumes.

__ _ __ _
~~~~. -,— -~~~~~~~~~~~
-- ~~~~~—~~~~~--
 31

5.5.7 Screen Model and Action History

In order to handle words such as inside and outside (e.g. ,
“pl ease draw a circle inside the triang le ”) it is necessary for KB to have
good know l ed ge of what is on the screen. For ‘outside ’ , for examp le , it
is n e c e s s a r y to f i n d space f o r t h e o b j e c t to be drawn . The s t o r e d
knowled ge a b o u t i t e m s in t h e s e m a n t i c n e t in some way p r o v i d e s t h i s
i n f o r m a t ion , but s p a c e — f i n d i n g c o u l d i n v o l v e i n s p e c t i n g e v e r y i t e m , and
much calculation . Consequentl y we would prefer to have either special
hardware to tell KB which lines or points are illuminated , or some sort.
of comprehensive model of the screen , possibly in a bit—map form. Such
a model could be used for space—finding, and discovering spatial relation—
sh ips and top olog i cal properties.
T o deal w it h questions of the kind “What d i d you d o ? ” and “Wh y did
you do?” , an action history should be included in the system , log ica l l y
in the KB module. This would consist of a t ime ordered set of actions
with their associated details. Use could be made of the simple indexing
mechanism available through set membershi p links. These would point to the
set of all objects of the same type , and they in turn would be linked via
a “ t ime line ” . This sort of mechanism could also he used to hel p with
an a p h o r i c r e f e r e n c e .

5.5.8 Functions
For positional indications such as “near ” , “ it ’, or “c e n t e r ” we
w ill probably need to have words that tri gger functions in tile KB modul e .
This has not been careful ly worked o u t v e t , b u t we feel tha t forms such
as (%POSIT . (NEAR (TOP #SCREEN)) ) or (LEFT
~tSCREEN )) could be ( U P PER

used . In line with our general princip le that as much informa t ion as
possible should be extracted from a sentence before KB gets to act on
it , w e feel t ha t QA w ou ld pr ob ably he the correct p lace to imp lement
such function handling. This would allow LA to pass areas or coordinates
to the MG section .

6. T Il E GRAPHICS MODULE
6.1 Introduction
T h i s m o d u l e a c c e p t s messages c o n t a i n i n g s c r e e n a I t e r , it Ion commands
and e x e c u t e s t h e m . F i r s t t h e message is inspected to see wha t t h e a ct ion
Is , and a m a r k e r i s set depending on whether it Is draw or e r ase . Then C lit ’

argument part of the message is exe c lit ed one step it a t ime . In the
current system onl y one step is present in each message (see- Appendix P

— ‘ —- - - - - — - --- -.-- ---— -.- - .- - - — _ _ _ _ _ _ _ _

 32

f o r CR message f o r m a t ) . The m a r k e r s e t s t h e u n d e r l y ing g r a p h i c s m a c h i n e

code r o u t i n e s i n t o e i t h e r draw o r erase mode. An o b j e c t is e r a s e d by
draw ing over it in erase mode.
The developm ent of a LISP/Plasma panel package (PLASUB) interface
enabled us to program the module entirely in LISP. The interface also
allowed the use of trigonometric routines from the FORTRAN Library, and
consequentl y we were able to do the angular calculations with ease. The
reason for the F4 prefix on the drawing function names is that at one time
these were to be written in FORTRAN .

6.2 The Screen Functions

The draw/erase functions consist of point drawing, circle drawing,
sc reen erase , and line drawing routines. There are four l ine drawing
r o u t i n e s , one for each sensible combination of the two endpoints , the
m i d p o i n t , an angle , or a length. The point drawing routine uses the
increment mode of drawing to draw a p o i n t c o n s i s t i n g of f ou r d o t s . The
l i n e r o u t i n e uses a b a s i c vector operation. Th e circle routine uses short
chord vectors to p r o d u c e t h e circle , with the angle step size that defines
t he ch ord v a r y ing depending on t h e size of the circle required .
The PLASUB package routines used in GR are:
ERASE — scr ee n era se
SETMOD — se t m ode o f panel
INCRE — incremental draw/erase
VECTOR — l i n e a r d r a w/ e r a s e/ m o v e
Working with the plasma panel involved us with three different
coordinate systems .

TOUCH PANEL SYSTEM PLASMA PANE l.

input system output

5 11 5 11 j K
~

x i -
. ic , ,
_.
~ II
~~~
511 511

~ ~‘ ‘~~ ‘ Us ’- . ~ f~
I,’ - ‘4
~~~ In- I1R .
~

F I G I ’R E C R —I
 __________ _______ — — -. —- — . — ‘--

33
6.3 P r o b l e m s and E x t e n s i o n s
The c i r c l e r o u t i n e is less t h a n p e r f e c t a t t h e moment , b u t t h e
authors arc aware that there are better circle drawing techniques that
c o u l d be used i f n e c e s s a r y . Because of t h e l a r g e n u m b e r of v e c t o r s
used in drawing a circle , and because of their frequency, the circle
drawing routine suffers occasionally from transmission errors , as the
panel is connected remotel y to the DECSystem 10. It may be that using
increment mode for circles would improve this.

—--- - - — -
~
-. , --
~~~ --~~~--
-- -
- .- - ------- - - - -~~-
 • ‘ r , ,, —r. . .- -
. ‘T~~~
~~~
-~~~~~~~~~~~~~~ ,---r .flT
~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~~ ~~~r ~~~~~~~~~~ !. ~~~~

34

7. CONCLUSION
7.1 System Performanc e
7 . 1. 1 General

The s y s t e m r e s p o n d s f a i r l y q u i c k l y to u s e r i n p u t , w i t h the only delay

b e i n g d u e to the fact tha t the terminal is remote from the computer and not
on the fastest pos ’~ible line . In consequence ther e is a substantial wait while
c i r c l es ar e be ing d rawn , aggrev ated by a sl i g h tl y inefficient circle drawing
routine and b y being swapped —out occasionall y. The sentence processing how-
ever , runs in approximatel y real time , with an average processing time of well
under a second from input to prototype formation. A slight speed—up of the
sYstem could be obtained by compiling the KB m odule of t h e system . This was
not compiled due to the fact that a large number of structur al changes would
have had to be made before compilation could t a k e p l a c e . Tile a v e r a g e pro—
cessing tLie for each sentence from inpu t to completion is about 2 seconds , and
depends on sentenc e type , sentenc e length , and screen operation. The number
of s e n t e n c e s tha t can be put on the screen is about fifteen , and the number
of KB—structure—buildin g sentences (drawing commands) that can be processed
without running out of space is about ten. It should be ncted tha t as a
spinoff from this project we now have a LISP—FORTRAN interface which may be
used for other LISP projects involving graphics . A version of LOGO has al-
r ead y been programmed u s i n g t h i s i n t e r f a c e .

7.1.2 Good Things

The t e c h n i que of m o d u l a r c o n s t r u c t io n u s i n g m e s s a g e p a s s i n g and a

c o n t r o l e x e c u t i v e proved to be v e r y successful . We were able to develop well
d e f i n e d i n t e r f a c e s , and p u r s u e d e v e l o p m e n t of m o d u l e s i n d i v i d u a l l y u s i n g
m o d u l e testing routines. Modules were initiall y built as “d ummies ” and
gradu al
l y built up to work correctl y. We were able to use d ummy modules Cc

build a complete system including , but isolating , -i newl y changed module.

The ability to monitor the messages in the executive allowed us to diagnose
interfacing problem s as well as errors within modules.
We were p l e a s e d w i t h t h e c c l at ive ease w i t h wh i cli it was possible to eonst ru e t
t h e system , g i v e n t h e use of t h e A E N and MENTAL pac k ;i ~~t ’ ~ , and w i t h t h e knowlcd ce
of a “ h a n d f u l ” of Al t e c h n i q u e s . We t a k e this te he an m d h e a t ion of the
p r o g r e s s of A l t e c h n o l o gy .

- • — .--- - --- ~~ --~~~~~~~,-

_ _ _ _ _
~~~~~~ ~~~~~~ . -
~
-
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~

35 ii
I

I) e s p i t e t h e l i m i t e d grammar , and t h e r e l a t i v e ly s i m p l e d e f a u l t s and

system heuristics , interacting with the system was p leasant , and its “in—
tell igence ” was very convincing . Several people who had had some Computer
Grap h ics exper ience reac ted f a v o r a b l y to demonstrations of t h e -vstem , and
were inclined (as even we were sometimes) to a’~cribe the system with gr eater
intelligence than it was capable of disp lay ing .
The abilit y to touch the screen with one ’ s finger proved to be a use-
ful addition to the stilted forms of language normall y investigated in
current research on Natural Language Understanding Systems. The processing
of d e i c t i c e x p r e s s i o n s , includ ing p lace adverbs and demonstratives , followed
directl y onc e touches were included. -

The use of the INFO function in the QA section of KB allowed the user to
specif y o b j ec ts b y using a partial descri ption. We f e e l t h a t t h i s is a power—
f ul techni que , and contributes a grea t deal to the ease of interaction with
the system .

7.1 .3 Bad Things

An overrid ing problem with the system was tha t , desp ite our best efforts ,
it tend ed to grow on its own . Some unplanned—for problem s inevitabl y occurred ,
and were solved , but due to constraints on tine and effort were not necessaril y
solved in tile best way. As usua l , there are hits we would like to rewrite.
It is not clear whether the simp listic world of lines , point s , and circles
truly prov ides a basis for assertions about more comp l icated worlds. ln
i d 1 i t i o n , we were a little suprised that even with just lines , p oints , and
c i :c les , and drawing and e r a s i n g , t h e system had to deal with qut te complicat e d
situations. Th is may be partl y a reflecti on of our own naiveté , and p a r t l y a
strong indication thla t even the 3imp l e s t of A l systems should not he u n d e r —
e S t im a t e ( h .
It would be d i f f i c u l t f o r us to c l a i m t h a t o u r w o r k r e s u l t e d in a
p r a c t i c a l sy s t e m . From a h a r d w a r e p o i n t of v i e w , a m a j o r w e a k n e s s is t h i t
t h e t o u c h panel does not a l l o w v e r y f Inc r e s o h u t i o n , and c o n s e q u e n t l v , p r ec i s e
p l a c e m e n t of o b j e c t s m u s t be done u s i n g c o o r d i n a t es . The sv ~- ’tem also lacks
rii nv of the t h i n g s t h a t one would expec t in a ( ‘input C r Gr ;ip hi i c s system , i , ,w

ever, t h i s is to he e x p e c t e d as i t wi s never our i n t e n t Ion to inip l , ‘w e n t -ii 1

‘1 I h o se f a c iii ties in the p i l o t pro t t

. - ‘-—-— —- — —-
.- —— -‘,---—
-_ -- -- - - -•-- - ------ ..
.- _ _ j
~~~~~
_ _ _
_
_
 t- s ---
~~~~ .a~ ~~~~~~
LrurJvr . r..
rz ~~7 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ — ~~~~~~~ .

36

More extensive modes of grap hical input would certainl y enhance the
communication in a graphics environment. Specificall y, one often would like
to simply sketch out an object rather than verball y describe it. Develop-
ment of a representation which would model the essential characteristics of
ske tches wo uld grea tly aid such an effort.
The physical management of the screen need s some improvement. Since the
user is permitted to draw anywhere on the screen , text and pictures often over-
lap. Providing a division in the screen between the drawing area and tile
area for text seem s to be the best solution . The text area could be at the
bottom of the screen showing onl y the last input and response.
The limited syntax of the pilot system is too prohibitive. A response
of “p leas e rephras e” doesn ’t prov ide the user with many clues as to wha t was
wrong with his input [Weischedel 977]. Certainly, an intelligent system
would at least be able to say that it had no knowled ge of squares i f asked to
draw one . Likewise , if the input contained all of the essential parts to
s p e c i f y d r a w i n g an o b j e c t , b u t t h e grammar d i d n ’ t p e r m i t the f o r m , t h e s y s t e m
s h o u l d o v e r r i d e t h e grammar .
We felt tha t as a tool for m anipulating kn owl ed ge MENTAL was at too low
a level. In a d d i t i o n to t h i s , the Brachm an—li ke formalism adopted for KB
uses fairly comp lex structures. This meant that a sim p l e insert or change
involved several o p e r a t i o n s . If we c o u l d s t a r t a g a i n , we would b u i l d a h i g h e r
level of language over MENTAL and use that for net manipulation [Bobrow 197611 .
We a r e aware that MENTAL has evolved , and t h a t newer v e r s i o n s may be s i g n i f i c a n t l y
e a s i e r to w o r k w i t h [ S h a p i r o 1976] .

7.2 Outstanding Problems

Languag e g e n e r a t i o n , as i t e x i s t s in t h e p i l o t NLC sy s t e m , i s qu i t e
weak. Althoug h the foundation has been laid b r developing th!s module , l i t t l e
e fort has been given to t h i s . N e v e r t h e l e s s , some v e r y s imp i e mechanisms h a v e
p r o v e d q u i t e e f f e c t i v e in a l l o w i n g an adequate leve l of communication. Lan—
guage is generated under the guidance of an A TN generative network w hich
sii ~~~~.s t s a tmp
e l a t e and a l i s t of substitutions. Thus , o u t p u t is l i m i t e d in
f o r m to p r e s i or e d temp l a t e s . T h i s r e q u i r e s tha t a l l sy s t e m r e sp o n s e s he a n t i c i -
to some extent when the templates ar
pated
~ designed . Also , th e knowled ge base

_ __ _ . •-
~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~ ~~~~
 ..
,-- .--
- —.~

37

r emains inactive during generation , u nlike in la nguage anal ysis. GEN should
he able to use the QA inter face to obtain additional informat ion to use durin g
g eneration , and it should also h ave a c c e s s to t h e luxicon used by LA.
There ar e a number of interesting but outstanding problem s connected with
th e KB module. Thle fir s t has alre ady been alluded to slig ht lv: that of
developing a full action history and model of the screen. The ability to refer
to past actions and their ordering in tine would allow a whole new range of
questions to be asked . It would also hlel p with reference problem s , as in ,

t I
f o r exam p le , “Now connect the other ends ” , or “Draw another ” . The screen
model , if it can in fac t be built , would allow heuristic p lac i ng of ob j e c ts
and use of heuristics such as those for “near ” , or “beside ” without undue
-
calculation.
¶
t

We believe tha t if knowled ge is to be collected together at all in a

sYstem then as much knowledge as possible should be included so tha t rich l inter-
connections may b e established . It is reasonable therefore , to include lexical
information in KB along with descriptions . For examp l e , the word “CIRCLE ”
and its associated markers could be stored with the concep t of circle in some
way, just as a print—name can be stored on a property—list. There is no
r eas on wh y the alp habe t shouldn ’ t have its set of concepts , so that the words
att ached to concepts would be made up of ordered sets of items which were
instances of letter concepts. Thus letters could be drawn on the screen using
the same general procedure as for obj ects. Fahiman [1975] makes a similar
poin t.
A problem that would have to be faced if the system were expanded is tha t
of erasing parts of objects. There’ are at least two subproblems. Consider the
f ig u r e produced by draw in g tw o ov erlapp ing triangles:

We will presume t h a t t r i a n g l e has been d e f i n e d ‘~~r t h e sy st e m . If t h e u se r

r e q u e s t s tha t p a r t s of l i n e s w h i c h a re i n s i d e a n o t h e r t r i a n g l e be e r a s e d , t h e
r e s u l t s h o u l d be:

-
/ ‘
\
-
.7\
_ _ _ _ - -~ -- . . . —~~~~ ---—______

38

Th is would involve some geometric calculation , and would leave parts of de-
fined parts of objects to be represented . This may be a representationa l
problem . A n o t h e r p r o b l e m o c c u r s if the user requests tha t the Inside tri-
ang le be erased :

The problem h e r e is t h a t t h e sy s t e m does not “know ” t h a t the o v e r l a p of t h e

two triang les i s i t s e l f a t r i a n g l e . A s i m i l a r s i t u a t i o n can o c c u r if several
obj ects are put together so that the spaces between them form some recognizable
shape.
The problem of obje ct sele ction br t o u c h i n g has been solved r at h e r

s i m p l y in t h e p r e s e n t s\’ster ’ b y sele ctin g t h e object i f the touch coincides

with its defined l o c a t i o n p o i n t , or , alt erna t ively , some other defined point.
Howev er , in a reali stic sy stem ofl’. would want to allow touches to other places
in o r d e r to s e l e c t t h e o b j e c t . For examp le , a c i r c l e c o u l d he s e l e c t e d b y
touching some point on its cir cul ’ti er en c e , instead of its center . It should
a l s o be p o s s i b l e to s e l e c t some c l o s e d f i g u r e l’v t o u c h i n g some p o i n t w h i c h l i e s
inside the f igure. A comp lete t h i e ~~ r v of obj ect sel e ct io n is a hard and ex-
tremel y intere stin y problem , and would he- a necos s - i r y addition to any practical
system.
Of til e pr oblems presented in s e c t ion ~~, b u t not discussed here ,
t h e mos t challenging are t h o s e of heuristic defaults and tile n e t w o r k speci-
fication of structural conditions. Bra chiman [l~~77] has more to say about the
l a t t e r pr oblem .

7 .3 Th e F u t u r e

This section describes some of t h e topics In whic h the members of our

group are c a r r y i n g o u t r e s e a r c h . We a re now using the N I ; p ilot sy stem ,-i s a
background f o r new id eas and w i l l he c o n c e n t r a t Ing on t h e t h r e e a r e a s of lan-
g u a g e ana l y s i s , k n o w h ed ge r e p r e s e n t a t i o n , a n d p r o t o c o l a n a l vs i s .

,- - - - -~~~~~~ - - -~~~~~ . - •- --
~~~ — - -- ~~~ - ~~~- - - - - - - •- - - ---- - - ~~~~~~~~~~~~ - -- ~~~~~
 39

Lang u age anal ys is in a practica l system should be robust. A practic al

h a b i t a b l e system should possess the ab ili ty to respond in te l l i g e n tly to sentences
regardless of the comp le teness of the grammar . Since a comp lete grammar of
English in this prac t ical sense doe s no t exis t , resear c h in na tur a l lang uage
unders tanding must therefore develop methods tha t account for all potential
input. Und er investiga tio n is the developmen t of a lang uage anal y zer which
co m b ines syn tac t ic wi th s eman t ic processing form ing a u nif o rm m odel wh ich
approaches this capability. It is believed that several additional benefits
will accrue from this approach. Tills scheme should be capable of correctl y
pr oces si ng many ellip t ical and - - e ungrammatical sentences. In addition ,
semantic cohesion should override poor syntactical form . This would allow
freq u en t us ers t o inven t shor t hand forms co nvenien t l
y wi th no al tera t ions to
the language processor . For examp le , in an NLG system one could use (17) and
(18) instead of longer forms .

(17) DRAW CIRCLE (200 ,300)

(18) LINE < touch> <touch>

W o rk in knowled ge engineering will attempt to improve and extend the

representation used in the pilot system in order to prov ide a knowled ge base
wh ic h us es a homogeneous represen ta t ion o f ma ny different kind s of information.
It should , f o r examp le , i n c l u d e the s tr u c tu r e of an ob ject , how to draw it ,
the object ’s function , and rela tio nsh i ps a m ong drawn objects. Tile knowled ge
base should serve as a lexicon , a database of facts , and a grap hical database.
In addition , a portion of the research w i l l be concerned with conversion between
th e different types of information , either by spontaneous computation or on
dema nd.
Protocol analysis should lead us to discover adequate vocabulary and
gramm ar and preferred modes of man—machine interaction , particul a rl y r a t i o s
of g r a p h i c a l to l a n g u a g e use and u s a g e p a t t e r n s . Some i n f o r m a t i o n a b o u t
s e m a n t i c s can a l s o he g a t h e r e d by r e c o r d i n g u s e r r e a c t Io n s ti s\’stein responses.
In a d d i t i o n to t h e e f f o r t in t h e t o p i c s a l r e a d y d i s c u s s e d , we a r e c o n c e r n e d
with system organization , language generation , gr a p h i c s , and processing drawn
input

—~~~~~~~~ -~~~ -~~~~~~-~~~~~~~ - -~~~~~~~~~ -~~~~~~~~~~~~~~~~~~ -“~~--~~ - - -

 40

7.4 Summary

Thi s r epor t has de sc r ibed a successful exper imen tal program for manipu-
lating simp le p ic tures on a computer graphics terminal. Natural language and
touch input can be used to give commands to draw , erase and name , and to pose
questions about the objects on the screen and the capabilities of the system .
We are confident tha t w i t h a robust and p owerf ul lang uage anal y z e r ,
m ...~tures of language and pictures for both input and output , s t ored knowledge
about the pictures being manipulated , and an inference capabil it y t o assis t
the user , we l
wl
i have a prod u c t ive res earch and app lica tions tool. Natural
L ang uage Grap hic s provides a fer t ile area f o r signif icant research and larger ,
more general NLG systems should result from our work.

.-
 —- -~~~~~~~~~

41

~~
. REFERENCES

Ag in , G.J. (1972) “Represen tation and Description of Curved Objects ” ,

Memo A IM— l 73 , Standford Al Project.

Badler , N.I .(February, 1975) “Temporal Scene Anal ys is: Conceptual Descriptions
of Object Movements ” , TR—8O , Depar tm en t of Compu t er Sc ience , Universi ty
of Toronto.

Bobrow , D.C ., & Winograd , T. (November , 1976) “A n Overview of KRL , A Knowledge

Represen tation Language ” , Rep ort No. STAN—CS -76—58l , Computer Science
Depar tm en t , Stanford University.

Brachman , R.J. (1977) “The Evolution of a Structural Paradigm for Repre-

sen t ing Knowledge ” , Ph.D. Disserta t ion , Harvard University.

Brach man , R.J. (Oc tober , 1976) “Wha t ’ s in a Concept: Structural Foundations
for Semantic Networks ” , BBN Report No. 3433.

Coles , L . S . (1968) “An On—Line Question—Answering System with Natural Language

and P i c t o r i a l I n p u t ” , P r o c e e d i n g s of 23rd ACM National Conference.

2ahlman , S . E . (May , 1975) “Thesis Progress Report: A System for Representing

and Using Real—World Knowled ge ” , MIT Al Labora tory, Memo 331.

K i r s c h , R.A. (August , 1964) “C omp ut er Interpr et a t ion of English Tex t and

Picture Patterns ” , IEEE Transactions on Electronic Computing .

Minsk y , M . (1974) “A Framework f or Repre senti ng Knowled ge ” , A l Me m o 306 ,

MIT A l Laboratory.

Neva t ia , R. (1974) “Struc ’ured Descriptions of Comp lex Curved Objects for
R ec og n i tion and Vi sual Mem or y ” , Memo A IM—250 , Stanford Al Project.

Palmer , S.i . (1975) “V i sual Percep t i

on and World K no wledge ” , in: Rum elhart
and Norman (eds.), Exp lorations in Cognition , W .H. Freeman Press.

Quam , L.H., & Diffie , W., “Stanford LISP 1.6 Manual ” , SAIL Oper ating Note 28.7.

Shap iro , S.C. (December , 1976) “An Introduction to SNePS ” , Technical Report Nc’ .
31 ( re v i sed), Indiana Univ e rs it y, Bloomington , Ind iana .

Sh a p i ro , S.C . (May , 1974) “ IC — MENTAL , A Working Paper ” , Computer Science

Department , Ind iana University, Bloomington , Ind iana .

S immons , R.F., & Benne t—Novak , C. (1975) “Seman t ically An a lyz ing an Eng lish
Subs et for the Clown s M icrowor id” , American Journa l of Computationa l
LinguIstics , m icrofiche 18.

Son d h e f m e r , N.K . (1976) “S p a t I a l R e f e r e n c e ’ and N a t u r a l l a n g u a g e Mac hine

Control ” , International Journa l of Man—Machin e Studi e s , V o l . 8.

We i se -h edel , R .M. (February , 1977) “Please Re—Phra se ” , Te chnica l Report No .

77/1 , Depar tment of S t a t i s t i c ’s and Conputer Sci en ce , Univers i t y of
Del iwa r e.

—~~- -~ - - -- ~~~- -
-- - - ---
~~~ ~~ —— -~ ------ - -- - ..
-- -- . —---— _ _ _
 42

Winograd , T. (1972) Under s tandin~ N a t u r a l L ua ~~~ , Acad emic Press.

Woods , W. A . , Kap lan , R . M . , & Nash— W ebber , B. ( J u n e , 1972) “The l u n a r

Sciences Natural Language Information System : Fina l Report ” , BBN
Report No. 2378.

W oods , W.A. (1973) “An Exper ime ntal Pars ing Sys tem for Transition Network
Grammars ” , in: R. R u s t i n ( e d . ) Natural Language Processing, A lgor it hml cs
Press.

-4

- -- ~~~~ - - --- ---—— - -- ~~~~ — .-—- ~~-

 - - -- a~~~~ r -_
~~~~ - -~~~~~~~~

43

4. APPENDICES

A. Module Sizes
B. S y n t a x of ATN G r a m m a r Sp e c i f i c a t i o n
C. Inpu t Vocabulary
D. Partial Lexicon
E. Sentence—Level Grammar

J
F. Touch Grammar
C. Noun Phrase Grammar
H. Prepo sitional Phrase Grammar
I. Question Grammar
J. Samp le Sen t e nces a nd t he ir Parse T im es
K. Descr iption of tile P r o t o t y p e
L. Lang uage Generation Temp la tes
M. Generat ive ATN Grammar
N. List of Ac t ions used in Semantic Network
0. KBNET — Initial State of KB Network
P. Tile CR Module Interface
Q. Syntax of Calls to the QA Section of KB Module

— f t M .rp .v :—1 rn t1 - rj - - - .
~~— — — — ~~~~~~~~~~~~~ ~~~ n r i i r ~ , ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~ . _ •
~~~~~ ~~~~~~~~~~ ~~~~~~ ~~~

I1
44

Words (Decimal )

ERR 26
EXEC 158
GEN
Tem p la te s 80
ATN N e t w o r k 359
LISP Code 199
GI 23
CR 297
IN 233
INIT 112
KB
MENTAL Fu nct ions 1153
I n i t i a l Network 3500
LISP Code 7000
LA
ATN Ne t w ork 5668
Lexicon 1927
LISP Cod e 2017
Misc . F u n c t i o n s 517

Appe nd ix A: Module Sizes

--- ~~~~~~~~~~~~~~ - - ---- ---~~~ -~~~~~~~~~ ~~~~~~ --

--
- ~~~~~~~~~~~~~~~~ , - . -
- --. - ~~
--~~~~~~~~~~
 — 7LU-~’ C -
~‘c ’u,-~ t- —-~
-—-— ~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

-E

45

1.. - - atm net - ::=

(--arc set - 1)
2. - -a rc set ~ ::(< state-- drc > 0)
3. - state- ::=
any ATN state name
4. - -arc > ::=
(CAT - category test - - a ct i on 0 <term act- -) or
(JUMP <state - test> :action> 0) or
(MEM <w ords - - <tes t> <action > 0 <ter m act’) or
(POP <f orm> <test > ) or
(PUSH — s t a t e - < t e s t > <p r e a c t ~~ < a c t i o n - - 0 < t e r m a c t > ) or
(TST < l a b e l - <t e s t > < a c t i o n > 0 < t e r m a c t ~~) or
(VIR <constit > <test > < a c t i o n > 0 < t e r m a c t - ) or
(WRD < w o r d s > < t e s t > < a c t i o n > 0 <t e r m a c t > )
5. —action > : := < s t o r a g e a c t - or < r e t r i e v e a c t’ or < c r e a t e a c t —
6. < s t o r a g e a c t > : : = (SETR <r e g > < f o r m > ) or
(SETRQ <reg-- <expr ession> ) or
(LIFTR re g -’ form- ) or
(L 1F TRQ <r e g - - < e x p r e s s i o n > ) or
(HOLD -z constit> <form> ) ~~
(PROTOTYPE — p r o t o case> — f o r m > ) or
<form
7. < retrieve act>::= (GETR <reg ) or
(CETF - feature- ) or
(RFEAT <fea ture> <form - -) or -
(CTGY — category” ) or
(NULLR < reg- ) or
(NEXT WORD) or
(PROTO <r eg-) or
(QACALL < q a f u n c t <qa a r g s > ) or
FEATURES or
* or
<form -
8. <create act> ::= (BUILDQ ‘-fragmen t - — .eg ~~~) or
(LIST < f o r m - ) or
(APPEND < f o r m - - f o r m ) or
(QUOTE < e x p r e s s i o n - ) or
(UNIONP --protc e pro to ) or
< f orm
9. < c a t e g o ry > : := (- - ct gy>- )) or < c t g v -
10. - c t gv - - : := an y I cx ic e I c i t c-gor y
iI. <test ::= - act ion - or F
12. <term a c t > : := (TO --st a t e - )
13. words - : := (- -w ords --)) or - - word -
14 . <word - ::= any l e x i c a l word
15. <preact- ’ : := (SENDR - r e g - —a orm - -) or
(SENDR Q r eg — - e x p r e s s i o n ’ ) o r
(! — f o r m ~ )
16. <label - : := -i L I S P a t o m
17. - co n s tit - : := a l I S P a t o m
18. < reg - : mv A’l N re i st e r narne
~~
19. — form - : : - ;R-t ion or any L I S P e x pr ~ - s- - i I ) - he I \ A 1 ~~eo t -2
20. - e x p r e s s in n - : := a ny L I S P exprossi oii
21 . - pr ot case- - ::= a c ; I s e - n a m e i n t h e 1~r o t o t v 1m
22. feature - :: my le - x i e - , l l e i t (i re-
23. -- q i f u n c t : : = I :IN DOR ,I or ( ONVE R T
24. — qa - i r g s - : := ir ~ iim -nt lIs t for qA
2 . - fra gment - : := .-i L i S I ’ s k i - i o t a I e xnre ~sston iis in 1 ~ * anti + (or s,,t’st i t it - ic
2 ( ~. -- proto - : := an N 1 2 p r o t o t v 1 ~c

AT N 2 timn o r Sp e i t h ps

— -
~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ - -
~~ — ~~~~~~~~~~~~~~~ -~~~ - — ——— — - -
~ ~~~~~
—‘
~~~~~~ ~~~~~~ --
-
~ ~~~~~~~~~~~~~~~~~~~~~~~~

46

A FOUR OR
AN FROM PLEASE
AND HALT POINT
ANGLE HAS POINTS
ARE HAVE PUT
AT HERE RADIAN
BETWEEN HORIZONTAL RADIANS
CAL L HOW RADIUS
CALLED HUNDRED SCREEN
CAN INCH SEVEN
CENTIMETER INCHES SEVENTY
CENTIMETERS IS SIX
CENTIMETRE JOIN SIXTY
CENTIMETRES KIND LY SMALL
CIRCLE LARGE STOP
CIRCLES LENGTH STRAIGHT
CIRCUMFERENCE LINE TEN
CM LINES THAT
CONNECT LONG THE
CONSTRUCT MAKE THEN
COULD MANY THERE
DECREE NE THING
DEGREES MILLIMETER THINGS
DIAMETER MILLIMETERS THIRTY
DID MILLIMETRE THIS
DO MILLIMETRES THREE
DOES MM THROUGH
DRAW NAME TO
EIGHT NAMED TWENTY
EIGHTY NAMES TWO
ENDPOINT NINE UNIT
ENDPOINTS NINETY UNITS
FRASE NOW V ERTICAL
FIFTY OBJECT CIIAT
FIRST OBJECTS W }IJ :RE
FIVE OF WHICH
FORTY ONE WITH

Append Lx C: input V o c ab u l a r y

-- - - - - - — - - _ - - - - -- - --— --- - ~~~

- - - _ _
~
- —.‘- — ‘~~~~ ‘~
_
~~~~~~~~~~ - - ---- - -- -- --- - - -- -
~~~~~~~~~~~ ~~~~~~~~~~ ~~~~~~~ ~~~~~~ ~~

47

/ = -~
= ~— -:
~~ ~

:
-

\0 +~~ -
~~~~~~ ~~
-

~ r -~~~~ ~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I
_ _

ANGLE
~~ N~~~~~~~~ -~~~~ ~~~~~~~~~~~~~~~ - -- ~~~ I
_ _ _ _-

~~ \RF UWI I~~ IS PL1 ~~~t

~~~~~~~T ~ 1
- - -
T TPREP ~~~~~~~~~ C~ i ) R I E N T ~~~LOCAT0R ‘

I (‘1~ -~I T )
~~~

BLT~JJ~~~~ R I L ~~~~~~~ F~~~~~~~ OI~~~ ~~~~~

—- —-- --4— --- - —-------------1------1

I I

-
N i— - - lIRI (
~ SIZE) - - ~~~~~~~~~~~~~~~~~~
LINE -1 _ _ _ _

-
~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
r
— —
III DECRFE C R11 NT
~ 1 IR I( \ ~I

:- — r
1~~~~~ - — - — ~~—-
I I ( _ ItI I
~ ~ —I 8 -
:
i:
- - -

i ~~i~~ï : ~~ :i
: 11
-

:
~~
T~~~~~ : TI_
~
~~~ E R~~ f -j } r
~~n~~~T n , N~~~~~~
- HFIJ
-
- T
r ———
( I’ c H l )
~~— 4—
—
~~~
— ~~~~~~~

—-
- —— — t
-
——— r
r -- _ _ _ _ _

t _ \I)\’
-~~~~~— —— - - -
FbI -:
— —

- — -
~~~ ~~~~ -
~~~~
_÷
( I:- i )
-

.I N I
~~~~~~~~~~~~~
II- ~~~~~~~~~
~
eH — - -
~
I 1
.

— — ——- —
-

el ( ‘I
- - — -
~ ~~ ‘ ~ ~~~~-

~~ ~~~~~ ~~~~~~~~~~~~~~~~ -- -~~ -

~~~LR~L a i
r—T T- (- J —_-,- - - i - - ----
‘

~ ~~~~~~~ ~~
III lb
I

--
Pb .> I N - N — - - - J_
- - I
— —
-

lb - I ~~~ (cIRCiT5~ - ~~
< -I
- ~~~
l J ) I
~~~~~~~~~~~~~~ _
‘~

- -
~ I
Ii i _ \ l l)( 1I
- — t0-I I) - - - -
—
I Ii CI I C5~ I
~~ I I ~~ I
T\l lI It\ I ) T (~~~~
~0 I~~~~ I I
~ 11 ( 1
4- - - . - - -
~~~~~~
N\ I I ) -
~
_ H I FJ _ ii —_-- :_
( -D I D
- - -
IlFi - 1. >I ) E F I — - -
4
-

~( l Zi FN DI ’oI N T ’ -
-
-

It 4:I II) ._ _ _ _ - - ___ 1 ,- ——— ___h_

-—-- -— —- - - - ------ -- - —— - — ‘ - - - - - ---—--—-- -
I

— ---- —

I
Apj ie-nd ix D: Part i a 1 [cxi c-oil

--
 — --—-.----- ---- ----—- - ,—,-
-— ~~~~“
r~~~~~~~ ’”~~~ —-V r~ t c — ---

,
‘ ‘ -

,
~~ ~~~~~~~~~~~~~~

48

P U S H P U S H

T O U CH

JUMP _ _ _ _

-r ~~~~~ A NV
J
TO P H A L T
~~~~~~~~ M~~~~~~~~~~~~~ S ST P

‘
STOP ’

WRO PUSH
ME PP

R OB J E C T
V P C
VI
VP HEA

c c
I I
0 PB U U L D

TST $ 40
PUSH .) 1k
NAME Q
TOUCH

PUSH v Ip ~~~~
p p
VP C NP

PB UI LO

V P C c

Append ix I- ~; Sentence—Level Grammar

_ _ _
— -----~~~~~ n -taL~~.
r~~~!~ ? ’
- -
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ‘~~~~~~~~~~~~~~~ —-—- - -- —- -- - - - . _ -- _;.;_ _
~ ~~~ ~~ —

49

TOUCH

V
; ~

t ~
ME M ((
TOUCHI ~~~~J TLEFT
-4
U)
-4

0
~
-o C
-I

( T ~~~~~)

TV
E T T

f 3 f
 - _ _ _ _ _
- -~~~ - ---~~~~~~~-— ~~~ ~~~~~~~~~ ‘

50

A p p e n d i x C: Noun P h r a s e Gr am m ar

(
~~~~~~~~~~~~~ Th4 7.
NPDETI
—1,

->
1~

NPDET

I —I

NPORD
C,
-4 -~

z
rJP NUM
-n
-4
-n
0 CAT
2. 1

a N P ADJ
U) 3 Mi
I-
2
2 CAT
)
CN N PR ‘

CLASSF
0

z
E
NP NP
0
L
c
0
Q

P0 S
NP b E NP N PI TB T NI A
COMP
a
NP

-- --
---
-- ----- _ _ -_ -a -
 —— --— —--—---__ - — ‘ — - —-- - ----- ---- --- —— -‘ — — c- - -=- —
— ~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~

51

~~ ( P P. P )
C C
U) ID
I I

P P C 2
U C
0
I
‘
P P P P

P P P P1

0
V

I
CASE PREP NP

Appendix H: P r e -p oC it in n il Phr ase - Grarn- 1,-,r

- — — - -- - — - — --
~~~ pppII
IIIIIIIIUIIUUIIUUIUIUUIUIIUIIUIWmrr .’‘ _ _
- - - ~~~~- -- .
--
~ :rt .. cfl&. . —.—_————---—--— — — --———
— —— --—-=-—.---- —

~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~
[ S Q4]
~~~~~~~~~~~~~~~~~~~
CSQID (~~~ Q2 çJ

~~~~~MAN~ (iii C Of)

@2 V01J)
*

cIi~3r
\
~
5, coP3 @THER ~~)

dl ~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
r’QPO P
~
I
( Q AS I ( R EO S )

~~~~
)PUSH NP
ç DCLPDI)
V
DC L ~~~~~~~~CL ~~~~~~~ DCL
~~~~~ ~ ~~

p p o To

A pp end ix I : Quot; t i n C r~mmma r

 A ppendix J: Samp le Sen tences and Thei r Pars e Times 53

SENTLNCES TIME (ms .)

Draw a straig h t l ime f r o m h er e ~ T> to t h e r e <T> 283

P l e a s e make f r o m here ~ T> a l ine to there -zT> 366
P l e a s e make me a l ine f r om (234 , 412) to here <T> 366
P l e a s e erase the l i n e here <T> 150
Draw a l i n e b e t w e e n h e r e < T - and h e r e <T> 300
P u t a p o i n t here <T> 116
Draw a point named FRED at <T> 217
Draw th e point ETHEL here <T> 200
Now connec t FRED and ETHEL 167
Draw a s t r a i g ht l i n e from - -T
thro ugh ~ T- - - 317
Join the point FRED with this point - T . - 266
Stop 17
Plea se connect this <T> with this <T> 266
Fr om here <T> draw a l ine to there <T> 333
From this point <T> draw a l ine to FRED 433
- T- -
T~ Draw a s t r a i g h t l i n e f r o m h e r e to here
- 917
With this - T - - connec t t h i s <T> 333
Draw a t h r e e inch l ine named L3 h e r e <T> 2 17
Dr aw a hor i z on tal l i n e w i t h a 3 i n c h l e n g t h h e r e <Tm. 350
Draw a vertical l ine 5 inches long called L5 to here -CT--- 317
Draw a circ le with a 2 inch radius here - T — 300
P l e a s e draw a h u n d r e d mm l i n e named DAVE at an ang le of
f ifty degrees here - T m .
Make me a circle called HEAD with a 2 inc h diameter there ~ T - - 266
Erase t h e l i n e named ARM 167

Erase the line from here -T - - to h e r e < Tm. 3 16

Draw a circle of rad ius f o u r u n i t s named EYE here -
~ T --
250
Erase t h i s c i r c l e - -1 - 133
C a l l t h i s - T o X43 150
Nam e the three inch line L3 21 6
What can you draw? 67
(an von draw circles? 100
Can you draw a c i r c l e w i t h a 3 i n c h c i r c u m f e - r e n c e ? 31 7
How many screen units is an inch? 2~ ()
Is t h e r e - a p o i n t named P 7 133
Wha t did you draw at 1’? 1 83
L iii von d r a w ? 100 C

_ _ _ - - - - -
~~~~~
--
~~~~~ - ---- -
-- --- - --—--
 r , .e -w -,r,s
~ r r .rif l r - -- - - j- —~ . .aene. -
~~
~~

54

<message - : : = (KB LA #Q (MG < p r o t o t y p e - ) )

<pro totype~- ::= ( (%TYPE . <type >
(ZACTION . <act> )
(%OBJECT . <obj> )
(%NANE . <name> )
(% 0RIEN T . <ornt> )
(%POSIT . <posit > )
(%SIZE . <s ize> ) )
<type - ::= IMP or Q

<act> ::= DRAW or ERASE or NAME

-z obj > :: = ST--LINE or CIRCLE or POINT

~name - ::= a LISP atom

<ornt> ::= a number in degrees

<posit~ ::= (<p t> 1)

-pt> : := (ENDPOINT < c o o r d s > ) or

(LOCATOR - c o o r d s > ) or
(TOUCH <coords >)

— coords -- ::= (<x—val- ’ . <y—v al> ) or

(NODE . <node i d e n t i f i e r> )

<x—va l - :: x—coordinate in screen units

<y—val > ::= y— coordiante in screen units

<node identifier — : : the identifier of a node in the semantic network

~size - : : = (RADIUS . <num ’) or

(DIAMETER . - n u m - ) or
(CIRCUMFERENCE . -~num - -) or
<num;-

<n um -~ ::= a length in s m e n u n i t s

A p p e n d ix K. : [)escrjj~flon of t lie Prototype

 ~ r~~~ --—---.~~~~~w~~~ -—
~ - -
- V r ~~~~~~~~~~~~ t
‘t
fl rr, r - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~ -

55

TEMPLATE
NUMBER TEMPLATE

TO 0
Tl (OK)
T2 (Thanks * for the session)
T3 ( N a t u r a l Language G r a p h i c s s y s t e m )
T4 (NLG — P i l ot Sys t em — summer , 1976)
T5 ( P l e a s e r e p h r a s e the s e n t e n c I
T6 (Object as s p e c i f i e d c a n n o t be t~~und )
T7 (*)
T8 (yes)
T9 (no)
Tl0 (I don ’ t know)
Tll (*)
Tl2 (* and *)
Tl3 (* * , F n d *)
T20 (at *)
T21 (* has no name)
T22 ( a * named *)
T23 (a *)

Appendix L: Language Generation Temp lates

- - --
~~~~~~~~ --
 —— -
—
~~~~~~~~~~~~~~~~~~~~~
— -- ---------—-----—
~
-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

56

M
S € MB IG ESH
~~~~~~~~~~~~~ ~~~~~~~ ~~~~~~~~~~~~~

F 0 P M S

ENA
~~ ~~~~
C-
C

TEMPLATE SUBS

A pp end ix M: Generative ATN Gramm ar

~~~~~~~- -
-- - - - - ----~~ - - - -~~—~~~~~~~ -- - - - _— - - - - - - --
- “_ _-- - - - - -
.— -- c-,-- -r-nr ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ !
~~~~?~~~~~~~ tr ~ - - -‘~~~~~~~~~~r -— - rar -
~~~ - ~~~~~~~~~~~~~~ ~~~~~~~ — ~- -- - - -

-
57

NAME PART OF FUNCTION

ACT— l POINT How to draw a POINT

ACT—2 ST—LINE Structural check
ACT—3 --
ACT—4 ——
ACT—5 ST—LINE How to find midpoint
ACT—6 ——
ACT-7 -—
ACT—8 ST—LINE How to draw a ST—LINE
ACT-9 -—
ACT— b ——
ACT— il --
ACT- 12 ——
ACT— l3 POINT Structural check
ACT— 14 DEFPOINT S t r u c t u r a l check
ACT— l5 —— --—

ACT—1 6 DISTANCE Structural check

ACT— 17 ANGLE Structural check
ACT-l8 -- ---

ACT—l9 CIRCLE S t r u c t u r a l check

ACT—20 CIRCLE How to draw a CIRCLE
ACT—2 1 CIRCLE How to find a locator
ACT—22 CIRCLE How to f i n d a r a d i u s

Appendix N: L i s t of A c t i o n s Used in S e m a n t i c N e t w o r k

~~~

- — ~~~~~~~~ -- -~~~~~~~~ --— - - — ---- -— - -

 - -‘- - r - -
~- ~r~
t~~~~~~~~~~~ —
— ~~~~~ ~~~~~~~~~ _ _ _ _ _ _ _ _ _ _ ________________________________________________________

58

Ap p e n d i x 0: KBNET — Initial S t a t e of KB N e t w o r k

((DEF INE
SUBSETOF SUBSETOF—
ELMNTOF ELMNTOF-
NAM E NAME-
TYPE TYPE—
ELMNTDESCR ELMNTDESCR—
DATTR DATTR-
ROLE ROLE—
RESTR RESTR —
ACTION ACTION-
ATTR ATTR-
INSTNCOF INSTNCOF—
VAL VAL—
CASE CASE—
NUMBER NUMBER—
OBJECT OBJECT—
ERASED ERASED—
STRUCTURAL STRUCTURAL-
1NSTANTI/eTES INSTANTIATES—
))
((BUILD TYPE SYSTEM NAM E SCREEN N ODEL))
((BuILD NAME OBJECTS TYPE SET)= XX)
((BUILD NAME LINES TYPE SET SUBSETO F *XX)= XY )
( (BU ILD NAME P R I M I T I V E S TYPE SET SUBSETOF *XX) = XZ )
((BUILD NAME DEFPOINTS TYPE SET SUBSETOF *XX ))
((BUILD NAME ANGLES TYPE SET SUBSET OF * X x ) )
((BUILD NAM E DISTANCES TYPE SET SUBSETOF * x x) )
((BUILD NAME CURVES TYPE SET SUBSETOF *XY))
((BUILD NAME ST—LINES TYPE SET SUBSETOF *XY SUBSETOF *xz))
((BUILD NAME POINTS TYPE SET SIJB SETOF *~~~~~ SUBSETOF (FIND NAME DEFPO INTS )

((B UILD NAME UNIVERSE TYPE SET

SUBSETOF— ~~~~~~

SUBSETOF — (BU iLD NAME SITUATIONS

TYPE SET
SUBSETOF— (BUILD NAM E CONNECTIONS TYPE SET)

((BUILD NAME DEFP O INT

TYPE CONCEPT
ELMNTDESCR — (FIND NAME DEFPOINTS)
STRUCTURA L ACT- 14
DATTR (BUILD ROLE XVALUE
TYPE DESCRIPTION
RESTR ( F I N D NAME DISTANCES)

DATTR (BUILD R O LE YVALUE

TYPE DESCR I PT I O N
RESTR (FIND NAME DISTANCES)
)
))
( ( B U I L D NAM E DISTANCE
TYPE CONCEPT -
;

ELMNTDESCR— ( F I N D NAM E 1Jli ~IAN LI -S )

STRUCTURAL ACT- 1 6 —.
))

_ _ _ _ _ _
_________________ _
- -— —- — — -——-- — — - - --—.-~~~~~ — .- . — —-- ——- —--—
~~~~~~~~~
-
~
—
~
- .: ----------
~~
— —: — — ---- - -
iuI!IIII
~
1

( ( B U I L D NAME ANGLE
TYPE CONCEPT
ELMN TDESCR- (FIND NAME ANGLES)
STRUCTURA L ACT— 17
))
( ( B U I L D NAME POINT
TYPE CONCEPT
ELMN TDESCR — ( F I N D NAME POINTS)
aIRUCTURAL ACT— l 3
DATTR (BUILD ROLE LOCATOR
TYPE DESCRIPTION
CASE %POSIT
RESTR (FIND NAME DEFPOINTS)

DATTR (BUILD ROLE TODRAW

TYPE DESCRIPTION
ACTION ACT— l
CASE %HOW

))
((BUILD NAME ST—LINE
TYPE CONCEPT
ELMNTDESCR- (FIND NAME ST-LINES)
STRUCTURAL ACT—2
DATTR (BUILD ROLE ENDPOINT
TYPE DESCRIPTION
RESTR (FIND NAME DEFPOINTS)

DATTR (BUILD ROLE ENDPOINT

TYPE DESCRIPTION
RESTR (FIND NAME DEFPOINTS)

DATTR (Bu iLD ROLE LOCATOR

TYPE DESCRIPTION
CASE ~ P~ii ~IT
RESTR (FIND NAME DEFPOINTS)

DATTR (BUILD ROLE MIDPOINT

TYPE DESCRIPTION
RESTR (FIND NAME DEFPO INTS )
ACTION ACT-S

DATTR (BUILD ROLE ORIENTATION

TYPE DESCRIPTION
CASE ZORTENT
RESTR (FIND NAM E ANGLES)

DATTR (BUILD ROLE LENGTH

TYPE DESCRIPTION
CASE %SIZE
RESTR ( F I N D NAM E DISTANCES)

DATTR (BUILD ROLE TODRAW

TYPE DESCRIPT I ON
CASE ~ FIOW
ACTION ACT- S

))
—U - ~~~~
-
~~~~~~~~~~~~~~~~~~~~~~~~~~ ~
—
~~~~~~~~~~~~~
----

~~~~~~ - -
~~~~ ~~~ ~~~~~~~~
‘
-- —- — ----—--- — -- -,---- ----
~~
—
-
- -- - ------- -- -

60

((BLILD NAME CIRCLES

TYPE SET
SUBSETOF (FIND NAME CURVES )
E LMNTDESCR (BUILD NAME CIRCLE
TYPE CONCEPT
STRUCTURAL ACT— 19
DATTR (BUILD ROLE LOCATOR
TYPE DESCRIPTION
ACTION ACT—2 1
CASE %POS I T
RESTR (FIND NAME DEFPO INTS)

DATTR (BUILD ROLE RADIUS

TYPE DESCRIPTION
ACTION ACT-22
CASE %SIZE
RESTR (FIND NAM E DISTANCES )

DATTR (BUILD ROLE CIRCUMFERENCE

TYPE DESCRIPTION
RESTR (FIND NAME DISTANCES)

DATTR (BUILD ROLE DIAMETER

TYPE DESCRIPTION
RESTR (FIND NAME DISTANCES )

DATTR (BUILD ROLE TODRAW

TYPE DESCRIPTION
CASE %HOW
ACTION ACT— 2O

))

STOP (*** TO FORCE WAY OUT OF MENTAL READ LOOP *** )

Note: The inverses of the links are indic a ted by the addition o a dash.
~
Note: The above descri ption is in t h e f o r m read by t h e MENTAL i n t e r p r e t e r .

- - -------
- -- --
~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~ ~~~-= --
-- ~~~~ — ~~~ ~~~~~~~~ --

61

Appendix F: The CR Mod u le In ter f a c e

The interface at GR is d e f i n e d as follows :

< message> :: = (CR KB i~Q ( <p r o t o t y p e > ))

<p r o t o t y p e> ::= ( (%TYPE . some typ e )

(%ACTION . some action )
(%OBJECT . some object )
(ZNAME . some name )
(%HOW . <program> )

-~ program> ::= (<step> )

1
<step> ::= (F4PNT x l Ll )
or
(F4L INE 1 xl vi x2 )
~~
or
( F 4 L IN E 2 xl ~~ ang le len ~~ j)~
or
(F4L IN E3 mi dx
~~~ x2 ~~)
or
(F4LINE4 m idx l e n g t h)
~~~~ ~~~~~
or
(F4CRCL cx cv radius )
or
(F4ERSE)
 ----- -- - ---
~~~

62

A p p e n d ix Q
The Sy n t a x of C a l l s to t h e QA S e c t i o n of t h e KB Module

<message> :: = (KB LA #Q ( QA < f u n c t i o n> ))

< function> : : = (INFO <keywords> < p r o t o t y p e form> )

or
(FINDOBJ <name> < t y p e >)
or
(CONVERT a number <units>)
or
(FINDD RAW < o b j e c t>

<keywords> ::= (<keys>

1
< keys> :: %OBJECT or %NAN E or %POSIT or NODE

prototype form> ::= ~~~~ valid c omp~~~te or p a r t i a l p r o t o t y p e

-

<name> ::= an obj ect name e x t ra c t e d f r o m in i n p u t s e n t e n c e

-~ type.> ::= ST—LINE or POINT or CIRCLE or NIL

<units -- : := UNIT or INCH or CM or ~1~1 or DEGRE E or RADIAN

<object- - : := a prim iti< e or sus -cted object

~~

----~~-- -- ~~~~ -- - —~~~~

 - — —-
___________ — ——- - —------ - -— - — - -- -----——‘-—-—-- - -- ---------—------ - —-----— --- --------.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~ ~~~~~~ ~— =-- >=‘-- - - : --- -O-= - - —----- - — - —- --.
~ ~ ~ ~

~~~

S ECU RI 1, CL AS ~~I SC A f l - ) N OF T~ - t P A- - 1 tA t Thts

,
I
- —
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

RE C T DOCUMENTA TION PAGE

~ [
fl ~~~~~ ()}>~~ ( 0 11 \ N
-
I. N ,<i~ $ — -.. A C L ~~~~~~ *~~~~ ~5’ —JI~~~E £, ‘ ~~ L 111~

7--~ii
~~~~~~~~~~ ~~~ . - N i VT
// ~ ~~~ ~~~

2\T ~~~1~~~’~-7
~~~~
~~~
-
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~

— —— ~~~~~~ r C O F
-
4. T iT L E - ,ot I S u h t , t i ~~) -
Ct E t>~O~R~ T S CR130 C C ,

\ \ ~~ l~~ R \ l ’ l l I L ~~ < I[ lrt cr i i

-
I-
(‘ - ‘
C -
~ ~~~~
.
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

_ _ _ _ _ _ _ _ _ _ _ _ ~~~~~~~~~~~ C H -; -T
~~~~~~~~~~~~~~~~~~ /
1- AU C — o -~

- \/.\ - 7 ~~ 23~~
- i d C / ~~~ n~ ,
I
f~~~

I
;
~~
~~ ‘s j- ‘
-

s. p -H tT~ T;T ~ H - - s u NA M E
~~~~~~~~~~

AND ADD RE SS
~~ ¶0. PROGRAM P5 EI TA > (
~~~- ‘t at e~ H.
LE N .
~
~ L ~~
n i v~~~~
er sitv AREA A n o~~
s ,c LOI T
~~~~ ~~
N . 0 5B E N
~-
Computer & I nfor mat ion Science Re >ca r ch Center 6 11C2F -

1/

-
C o l u m b u s , ~~l 1 L32 10 --

~
-
-. ~~~~~~ /
~~~~~~~~

-
23 OLJ A 2
I C-N ’ S’ O L LI NG O F F I C E N° M E A N D A D D R E S S
~~ ~~!
~~~~~~~~
D AT E
~~~~~

Air F or c c ~~fN c e of Scientific Research /Nm un

1/’
~ ~~ - - -

-
¶
e ii
- - - -
[3~~ i i i n i ~ tt i)i - ~~~~~~~~~~~~~~~~~~~~
— -- •-
_ _‘ - -
~ ,—
~~~~ \

4 M O N I T O r I N G A 3 ~~ \ Cy C~~ ME & A C I I U If or 1 o I It ~ Oil )

U N C L \ S S I F 1 LI
--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SCHEDULE

5 U I C C u I R I IT I O N S T A T E M E N T ‘o f t h i s Re po rt )

A p p r o v e d f o r p u b l i c r c l c ) 1 s e ; I l i s t r i b u t i o n u n l i n .j t e d .

17. DI S T R I B U T I O N ST A T EM A N T ‘o f the ~ h s t r o c t ~nr, r-- I in i

l , A ~ u . ‘I IS! I I
~ - -- - - !
~

iS. S UP PI S - C’A PY NOTE S

1) XC I W O R D S (O t t . on p- - . — out, if ess - n r II It 0-I - numbe r)

- -
~~~4 T
2) -~~- - 0- - -
~~~
I S r i -p o r t dt- ~;c r i h - s in o;- .~ ”- I I - :- - tit - ii svi-i t en I or dr aw l si op Ic
l et ti r e-s 011 ,i i i l ilt I - - r cr ;l ph i c ; mu u s I ni~ n i tin II I - iui ~ . t t i I C I
~~
flhI

SVS Ciii Is ,i .ihle Of ‘ hi I ;c iii I 0~~~C po in t ~ • 0:1 I c I (ii Ofl I - ‘01 10.1 id r cii hi -

i-; ,-r , i-s wi I i~ i On- ;w inc quest b u s ih - ’ o1t -, .Ite:-1 &.hI I i % I it t i N .101 ob ~~ -e -
~

1 he scr- - n. Er .i.str --s are ’ ii SO pe -ICi t t II — l. i fllI i IYC i ul 3 CI i t c iii 1- - em—

Ire - h I i ; H - b with t ouc h- s t , I o ; 1Vt -V p o s i t i -’ n a l inl in - it b - ui . - - -~~~~~~~

DD 1473
“ ~
r ii , IN O~ ¶
~~~~~~~~~ - O ’ S O L E T F
~~~~~~~~~~~~~
I • —
~~~~~~~
~ I~~ 1)
I 7:-
~~~~~ ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~
— - - _ i11 j
 —I- - ~~~~~~~~~~~~~~~~~~ .3 ~~ ~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ —

i .

S E C U R I T Y C L A S S I F I C A T I O N OF T H I S P A O E . Cf7 eo f~ . ’ . di

2~~. ,- \ B S T L A C I —

i r - :pleme nN .-d h’~- t he a u t h o r s I - t r i n i ~ Summer

The - i v - t c m w;os desi gned and 1 9 7~
was w r i ttA-fl in LISt1 1.6 , runs in ih ~~’ I 1 t -~!IK ~-c rds on t DECSvst em— 1 I)
-O Cl t ’u i and di sp l ay s p i c t u r e s on an :i c’s~ P1 i.si~.i Paree 1
or ,

The sys to:- : was im i - l. L-me u t c - d to t e s t C O t i d~ - ;Is on - y st v-rn o r g a n i o o t i o n ,

to e st a b l i s h t h e v i a b i l i t y of c o m b i n i n g l . u I I c u : S C C -m d ~ r ap b i c s , and t o
e x p e r i m e n t w i t h a p p r o p r i a t e A. I - techniqu es.

I :
~~ E .-‘,S~— I I- I l- i)
SCCU RITY CLA$$IFICATIO Or .C P A c ~E ’W h e n Data EnI.r.dO
- - _______