Intelligent natural language interface for a signal
processing system
D.T. Morris, PhD
D.E. Asumu, PhD
Indexing terms: Signal processing, Interface states
Abstract: An intelligent natural language interface
to a signal processing system is described. Users
type commands to the interface in a form of
english orientated towards mathematical oper-
ations. The interface will interpret them, and if the
commands are known to the interface, it will
execute them. If any commands are unknown to
the interface, a dialogue will be initialised whereby
the system will seek clarification of the unknown
ones. Such commands may be re-expressed either
in terms of previously encountered synonymous
commands, or as functions of simpler commands.
The newly defined command is saved for future
use. Nonprogrammers are thus able to define
‘programs’ and tailor the signal processing system
for their own purposes. Although natural lan-
guage interfaces are not new, this system is novel
in its learning capabilities.
1 introduction
To issue instructions to a computer using a natural lan-
guage interface one would type a command in a natural
and free form rather than using a predefined syntax. For
example if a user wished to add two numbers he could do
so by instructing the interface to sum the two numbers,
add them up, print their summation and so on. The
natural language interface frees him from the restrictions
of learning and adhering to a closely defined command
syntax.
Many authors have described natural language inter-
faces to computer systems of various kinds. Ballard [l]
presented a system designed for matrix manipulation
which also contributed to the operating system interface
of Fink et al. [2]. Heidorn [3] presented a review of four
natural language interfaces to automatic programming
systems. These varied from interfaces which simply
altered the values of parameters in prototype pro-
grammes to a more ambitious automatic programming
system which generated a new code (with limited success).
Much interest has also been shown in the realm of
interfaces to question answering systems. Woods et al.
[4], Plath [SI, Waltz [SI, and Warren and Pereira [7]
have all produced similar looking interfaces; similar in
the sense that one typed a query which the system trans-
lated into a relational database query form which was
Paper 7311E (C2, C4, E16). first received 8th May and in revised form
3rd November 1989
The authors are with the Department of Computation, UMIST, PO
Box 88, Manchester, M60 lQD, United Kingdom
I E E P R O C E E D I N G S , Vol. 137. P t . E. N o . 5. S E P T E M B E R 1990
then executed. The reply was formatted by the interface
into a suitable textural or graphical output. The systems
differed in the formalisms used to translate the user input.
Woods et al., Waltz, and Ballard all used a form of graph
traversal algorithm (the graph was termed an augmented
transition network) which will be examined in detail
below. Plath used an algorithm with which the ordering
of phrases in a sentence could be altered without chang-
ing the sentence’s meaning. The reordering was continued
until the sentence matched a template. Warren and
Pereira translated the query sentence directly into a
Prolog expression by searching the sentence for so-called
determiners (the words a, the, some, all, and so on). Each
determiner had an associated predicate with an appropri-
ate number of argument slots which were filled by a
further search of the query. The predicate expression was
then optimised and executed. Warren and Pereira termed
their solution to the syntactic stage of the understanding
process, the extraposition grammar.
A system called LIFER [8] has been developed for the
construction of natural language interfaces. The devel-
oper must define a grammar for the system using a modi-
fication of the augmented transition network - the
transition tree - in a programming language (InterLisp).
Each input sentence defines a unique path through the
tree, code for executing the user’s requests may be bound
to the tree’s leaf nodes. It will be seen that the system
described below has much in common with LIFER.
Other than explicitly defining them, LIFER makes no
provision for representing synonymous sentences.
The processing of natural language in all these systems
may be divided into three stages. The syntactic pro-
cessing stage which parses the input sentence; the seman-
tic processing stage which ‘understands’ the parsed
sentence and passes instructions to the computer; and the
output stage which will interpret the results returned by
the computer and format them in a suitable form for the
user. Difficulties arise in the syntactic stage owing not so
much to the range of vocabulary that the user may
employ as to the large number of ways in which the same
command may be expressed. Maintaining a record of all
possible ways of formulating all commands would
require a prohibitively large amount of storage space.
Two formalisms are most widely used in overcoming this
problem : Chomsky’s transformational grammar [9] and
Wood’s augmented transition network [lo]. In
Chomsky’s formalism a parse tree is produced which
reflects the grammatical structure of the sentence. Rules
have been formulated which allow the tree to be trans-
formed without the meaning of the associated sentence
being altered. These rules are applied until the parse tree
matches a sample sentence. This algorithm was used by
Plath. The augmented transition network (ATN) is a
771
finite state transition graph whose arcs are labelled with
the input which will trigger a transition between states.
Using either of these formalisms the semantic processing
is easily achieved by binding a sequence of instructions
either to the sample sentence or to the final node of the
ATN.
To be universally useful the natural language interface
would have to ‘understand’ the entire language of the
users, including all specialist words and phrases. It has
been pointed out [ l l ] that this goal is at present unat-
tainable owing to the amount of knowledge required by
such a system. Any natural language interface will there-
fore be restricted in the vocabulary it knows and is conse-
quently restricted in its sphere of usefulness. Such an
interface should then be properly be termed a pseudo-
natural or restricted natural langauge interface. The
interface would ‘know’ a basic vocabulary plus the
specialist vocabulary of a restricted number of spheres of
application. Two forms of language restriction may be
identified : owing to the interface’s knowledge of grammar
and vocabulary. The interface’s knowledge of grammar
and the size of its vocabulary may be related, for example
a grammatically knowledgeable interface could recognise
that ‘addition’ and ‘add’, and ‘summation’ and ‘sum’ are
effectively synonymous whereas an interface with a poor
knowledge of grammar would require all four words to
be stored as synonyms in its dictionary.
The restricted natural language constitutes a set of
commands which must be adhered to as must any set of
predefined commands. The restriction imposed by choos-
ing a subset of language has imposed some form of
command syntax: in the example quoted above the inter-
face knew about adding and summing numbers, but
would not necessarily know about the addition of
numbers. However, if the subset of understandable com-
mands is sensibly chosen, then the user may be unaware
that the interface he is using is restricted in any way. For
example, in computing averages one seldom talks of the
first moment, this phrase could therefore be omitted with
little or no loss of usefulness.
If the interface is given the capability of learning new
commands, the language restriction is effectively removed
since the learning process would be invoked whenever an
unrecognised command is encountered. We may there-
fore envisage an interface having a sensibly chosen,
restricted vocabulary and the ability to extend this
vocabulary should unknown phrases be encountered.
In using a natural language interface a significant
amount of processing is involved in recognising the user’s
commands. This processing may make systems using an
interface somewhat slower in their response than the
same system without the interface. Whilst this may be
frustrating to the experienced computer user, the naive or
irregular user may be prepared to tolerate it as the cost of
being able to communicate easily with the machine. It is
not suggested that natural language interfaces should at
present provide the normal means of communicating
with the machine, rather they may provide a vehicle for
the inexperienced or naive user to communicate ; experi-
enced users may still prefer to issue commands directly. It
is highly likely that this restriction will be removed in
future as more powerful processors are produced.
This paper will describe a pseudonatural language
interface designed to allow naive computer users to
define and perform signal processing tasks. The intended
users of the interface would be knowledgeable about
signal processing, but ignorant of any programming lan-
guages. The interface is therefore biased towards under-
173
standing mathematical terms and programming
methodologies. The interface is able to learn new com-
mands; if the user issues a command that is not recog-
nised by the interface, then a learning dialogue is
initiated. The new command is stored for future refer-
ence. Despite this bias, the system has been implemented
such that transitions to other applications will require a
minimum of reimplementation.
The algorithms used by the system will be described
under the headings of syntactic processing, semantic pro-
cessing, the formatting of output, and the teaching of new
commands. Examples of the system’s use will be given.
Finally, the application specific aspects of the interface
will be discussed. Since the subject matter of this paper is
of a multidisciplinary nature, a certain amount of tutorial
matter is included, where required this is presented at the
beginning of each Section of the paper.
2 Syntactic processing
Syntactic processing is the identification of an input sen-
tence’s component grammatical constructions. As
described above, two representations are widely used for
this purpose. One of these, the augmented transition
network has been used in the present application: it will
be described in detail.
A block diagram of the syntactic processor’s architec-
ture is given in Fig. 1. It has four main components:
user input
a bbrev i at i o n ex pans ion
spelling checker
m
current input
ellipsis processing
a t n traversal
parse succeeded
Fig. 1 Block diagram of architecture of syntactic processor
the preprocessor, the sequencer, the ATN parser and the
ellipsis processor. These components will now be exam-
ined.
The user’s input is preprocessed in two respects. Any
abbreviations are expanded and the text checked for
spelling errors. The system requests the user to verify the
spelling of words not appearing in an on-line dictionary.
The preprocessed sentence is then passed to the
sequencer. In defining the interface it has been assumed
that the top level construct employed in formulating
commands is the sequence. A sequence is a connected
group of natural language statements each referring to a
IEE PROCEEDINGS. Vol. 137. P t . E. N o . 5. S E P T E M B E R 1990
single action to be performed. The sequencer’s function is the operator and operands. The system is not restricted
to establish the boundaries between statements. to this: infix notation may also be used. Neither is the
The statements of a command may be connected by system restricted to having operators written in full, since
the words ‘then’, ‘and then’ or ‘and’, or the colon symbol. it is being deliberately biased towards mathematical
Statements sequenced by any connector other than ‘and’
are easily identified since these symbols are not used in
any other context. Ambiguities may occur when ‘and’ is
used to connect statements since this word may also be
used in data definitions (‘add 2, 3, and 4‘). Such ambi-
guities are resolved by searching for the longest self con-
sistent substring within a command.
As an example of the sequencer’s operation, consider
the following input command ‘consider the berlin eeg
data: display the samples between 1.5 and 2.7 milli-
seconds in the last three channels and then multiply this
by 0.75: divide this result by 2.43 and display it’. The
sequencer would first search for any unambiguous con- please ” p r I n t the”
nectors (‘:’, ‘then’, ‘and then’) and mark their positions.
Any of the statements thus delimited containing the word
‘and’ are further processed by attempting to use the ‘and‘
in a data specification context. Should this fail the ‘and’ is
treated as a connector. The above input would therefore
yield the following separated statements:
“add”
consider the berlin eeg data
display the samples between 1.5 and 2.7 milliseconds
in the last three channels
multiply this by 0.75
divide this result by 2.43
display it.

The sequenced statements are then passed to the ATN

parser which attempts to parse each one using each of
the available networks in turn. As an example of how the
parsing proceeds, consider the command to add two
numbers. This may be specified by the ATN of Fig. 2. In
this example there are two arcs leaving the initial, addi-
tion node. One arc terminates at the initial node and is
labelled ‘pleasantry’; the other terminates at the node N1
and is labelled ‘instruction’. Therefore, the initial segment
of the current input must satisfy either the ‘pleasantry’
and then the ‘instruction’ portions of the ATN, or the
‘instruction’ portion alone. Phrases such as ‘please’, or
‘print the’ will satisfy the pleasantry network; verb
phrases such as ‘add’, ‘sum of‘, ‘addition of‘ will satisfy
the instruction portion. Recognising one or both of these
phrases will advance processing to node N1. Having
reached this node a segment of input satisfying the
‘number’ ATN must be found. Any integer with at least
one digit will allow this. The relevant portion of the ATN
is easily altered to allow other types of number. The
number ATN also illustrates the recursive properties of
the network in that the label of an arc in an ATN may
refer to the ATN itself. The number recognised by
parsing the number ATN is stored for future use, and
processing passes to node N2. (It is the association of an
action with an arc of the network that augments the
network.) At node N2 an optional number or the literal
word ‘and’ may be recognised. The scan of the input will
only terminate successfully if the input contains the literal
‘and’ followed by one integer; note that in this ATN
there is no provision for the word ‘and’ in any other posi-
tion in the sentence. Having reached the terminal node of
the ATN (node F ) the procedure associated with it may
be invoked; this will simply add together the numbers
that were stored during the traversal of the ATN. In this
example note the potentially ambiguous use of ‘and’
referred to above.
In the example, prefix notation has been used to define
IEE PROCEEDINGS. Vol. 137. Pt. E. N o . 5. S E P T E M B E R 1990
number
‘0”
number
“9”
Fig. 2 An example augmented transition network for the representa-
tion of requests to perform addition of two or more integers
operations the mathematical symbols are also allowed. A
hierarchy of operators is enforced : parenthesised expres-
sions are evaluated with the highest priority, followed by
multiplication and division with equal priority, then
addition and subtraction. Operators of equal precedence
are evaluated reading from left to right.
Two forms of data selection are commonly used. In
one a label is given to a specific set of data: in the present
application data is divided into channels labelled one to
twelve. Data may also be explicitly labelled ‘. . . and call
the result X’. In the other form of selection, data are
extracted using a predicate: ‘all samples greater than 1.5
millivolts’ for example. The first form of data selection
may be termed passive and the second active. Active
selection requires that data be present from which to
select the points of interest. This data would be initialised
by passive selection.
The selection process may be represented by the ATN
of Fig. 3. This allows commands to be applied to selec-
tions such as ‘the points with amplitudes greater than 2.5
in the second channel’. Selection of commands may also
be performed using this network : the required command
is applied by actively selecting data for it.
377
 The repetition ATN must recognise the lists of com-
mands to be repeated, the data to which the commands
are to be applied, and an optional termination test. In the
current application the list of data is usually specified by
selection
selection
passive
Fig. 3 Selection transition network
passive selection. Using the networks already defined,
repetition may be defined simply by the additional
network of Fig. 4. This is merely an implementation of
“repeat”
keyword
termination
U
‘until ’ a (3 test
Fig. 4 Repetition transition network
the repeat . . . until looping structure. It has been
observed [2] that naive users quickly discovered the exis-
tence of a nonspecified structure in an interface when its
use was required. Note also that repetition may be
implied by the time sequence nature of the data, for
instance when a request is made to apply an operator to
a subset of the data.
Whilst these programming primitives are simple, it is
the thesis of structured programming [12] that the three
constructs (repetition, selection, and iteration) are suffi-
cient for defining any algorithm. Therefore, although the
range of programming primitives is limited, it is suffi-
ciently large for any program to be implemented.
The primitive networks described thus far, and any
networks taught by a user are stored separately but are
treated identically when the system attempts to match
them with user input. This separation has been enforced
to maintain the generality of the system since the meta-
networks (the repetition and selection ATNs) will be
appropriate for all programming applications, the
command ATNs will be appropriate to mathematical
314
applications, and the user defined and data specification
ATNs will obviously be of very limited general use. In
parsing input the system attempts to match it with net-
works in decreasing order of specificity.
The third portion of the syntactic processor deals with
ellipsis. Ellipsis is defined as the omission of a word or
phrase which is necessary to the sentence construction
but whose identity may be inferred from the sentence’s
context. For example if the user had typed the command
‘add 5 and 6’ and he followed it by typing ‘multiply’, the
ellipsis processor would expand this to ‘multiply 5 and 6‘.
When using the transformational grammar, expansion of
elliptical sentences could be achieved by replacing with
the current input a portion of the previous parsed input
having the same syntactical label (although this may not
allow unambiguous expansion). Elliptical sentences may
not be resolved using transition networks, therefore a
simple, representative, independent ellipsis resolution
algorithm has been developed for the present application.
The system’s ellipsis processor is invoked if parsing of
the current input fails and the current input has a lower
wordcount than the previous input. This processor will
attempt to substitute the current input for strings of
equal wordlength at all positions in the previous inpilt.
Meaningful sentences that result will become the current
input and the system will attempt to parse them. Whilst
it may seem that this algorithm will produce an inordi-
nate number of candidate sentences, this has not been
observed in practice. A multitude of candidates is only
generated when the user has not provided sufficient infor-
mation to specify a command unambiguously. For
example if the user was to input ‘add channels 5 and 6‘
followed by ‘6 and 7’ parsing would fail and the ellipsis
processor would be invoked. The only grammatically
correct sentence that the algorithm will generate is ‘add
channels 6 and 7’. This will then be processed. (Whist
‘add channels 7 and 6’ is also a grammatically correct
sentence, the algorithm will not generate it as the word
ordering is not preserved.)
Pronouns in a command are always assumed to refer
to the result of the previous computation, rather than to
the same data as the previous command. Thus one could
type the command ‘multiply the data by 7’ followed by
the command ‘multiply it by 3’. The net effect is that the
data has been multiplied by 21 rather than by 7 and then
3. Enforcing this identification allows us to place all
pronouns on an arc of the data networks and associate
the previous result with them.
Data in the input is specified by noun phrases. These
are highly domain dependent. For instance, in the current
application, data consists of files made up of twelve sets
of time series measurements called channels. The chan-
nels in a file contain the same number of data points
which may not be the same as the number of points in
any other file. The current system will understand data
specified as points identified by their number, the time at
which they were collected, a range of these values, or the
data’s channel number. Changing the system’s domain
will require extensive redefinition of the data specifying
noun phrases.
3 Semantic processing
Semantic processing is achieved using a frame based
approach. A frame consists of a collection of labelled
slots. Each object in the system, be it a command or data,
is represented by a frame. As many slots as possible in
each frame have been given default values, either data or
IEE PROCEEDINGS, Vol. 137, Pt. E, N o . 5, S E P T E M B E R 1990
references to subframes. As a command is parsed the ins-
tructions augmenting the network cause a frame appro-
priate to it to be instantiated and the frame’s slots are
filled or altered. When sufficient slots in a frame have
L
channel 6
Ih Isubtract
E
i
channel 9
t
channel 7
Fig. 5 Frame structure associated with the sentence ‘multiply the sum
ofchannels 2 and 6 by the difference between channels 9 and f
been filled and parsing of a command is completed, the
appropriate action may be performed. Rules have been
formulated for frame handling. If the newly filled frame is
a subframe its result is transferred to the parent frame
and the subframe may be deleted. If the current frame is a
command frame, filling its slots will result in the
command being executed. The result of executing a
command may be passed on as data to a further
command or may be stored in the system’s workspace
frame for possible consideration by further input.
The system’s workspace frame contains four slots for
the current and previous inputs and results. The previous
input is retained for ellipsis resolution. The previous
result is maintained and displayed for the user’s benefit.
The workspace frame also contains slots for storing
labelled portions of data.
As an example of the semantic processing, assume that
the user had typed the command ‘multiply channel 4 by
7’. The syntactic processor will parse this and cause the
multiplication frame to be instantiated, since ‘multiply’ is
recognised by the multiplication ATN. The multiplica-
tion frame will have an indeterminate number of slots,
referencing data subframes. Further parsing causes the
I E E P R O C E E D I N G S , Vol. 137, P t . E, N o . 5, S E P T E M B E R 1990
operands ‘channel 4‘ and ‘7‘ to be extracted from the
input and placed in two of the data subframes. The data
subframes are now replete, the data may be moved to the
parent frame and the subframes deleted. Sufficient argu-
ments are now present for the multiplication to be per-
formed, the result is stored in the workspace’s current
result slot since there are no further commands to be
considered. The multiplication frame may now be
rem0 ved .
As a more complex example consider the input sen-
tence ‘multiply the sum of channels 2 and 6 by the differ-
ence between channels 9 and 7’. The main verb of the
sentence is recognised as multiply, the multiplication
frame is therefore instantiated. The arguments for the
multiplication frame are recognised to be the results of a
summation and a substraction. These two frames are
therefore instantiated and linked as subframes of the
multiplication frame. Further parsing identifies channel
references as the arguments to the two subframes. The
frame structure of Fig. 5 is generated. Having filled suff-
cient data slots for processing to proceed, the system will
perform the instructions specified by the lowest level
frames, transfer the results to the parent ones and delete
the lowest level frames until the top level frame’s result
has been computed. The result is transferred to the work-
space.
4 Formatting of output
In normal use the interface will accept instructions which
will be displayed, interpreted and executed. The results of
the computation will be displayed in an appropriate
manner. The previous instructions and result are also dis-
played to enable the user to quantify the effect of a
command.
The interface produces several types of output. It
repeats the current and previous commands, one state-
ment per line in the case of multistatement inputs. It
traps any error messages and prints them, rather than
allow them to crash the system. It outputs the results of
computations as a time series display, or as a histogram.
It also enters into a meaningful dialogue when requesting
clarification of a command. This last aspect of the system
will be examined in the next Section.
5 Teaching the system
Should all attempts at parsing the input fail the interface
will require clarification of the unknown portions of the
input. The system will search the current input for vari-
able names (anything satisfying a data specification ATN)
and replace it or them with formal parameters. The input
command is then formatted into a request for clari-
fication. The user responds by typing a definition which
the system will build into a new transition network.
Parsing of the input command continues in a depth first
manner (the most recently input command not yet under-
stood is parsed, and if necessary clarified). User defined
networks are stored separately to the system-provided
networks since this aids the maintenance of user speci-
ficity.
One aspect of system efficiency has not been
addressed, that is the response of the system to being
taught a command it already knows. Ideally the system
would attempt to match the newly expounded definition
against all the previously taught definitions. If a match is
found then the newly defined command would be made a
375
synonym of the matching command and the new defini-
tion discarded. This contraction of the system’s data base
could take place either during the learning phase or as a
separate activity. In the current version of the system it is
intended to be a separate activity and has not yet been
implemented.
As an example of how the system is taught new com-
mands consider the following dialogue. The user has
typed the command ‘remove the average from channel
one’. The system is unfamiliar with the command remove,
it therefore requests clarification:
clarify REMOVE (x-1) FROM (x-2)
The terms in angle brackets have been inserted by the
system in place of the noun phrases used in the
command. The user is required to respond with a defini-
tion, assuming that ‘remove’ had some meaning peculiar
to the application being considered:
scale x-1 by 0.5 :
decrement this by 1.2:
subtract this result from x-2.
The system will recognise this as a sequence of com-
mands and will bind them to the ATN it has extracted
from the original command. It will now attempt to parse
each command. Assuming it is unfamiliar with the first
two; subtraction is a system primitive and will therefore
be recognised. The user may redefine scale in terms of
multiplication which will be recognised as a primitive.
Similarly decrement may be redefined in terms of sub-
traction. The three new definitions are saved in the user’s
command file.
As an example of learning a more complex command,
consider convolution. This may be defined as performing
an overlap integral between a signal and a template for
each point in the signal. The overlap integral is defined as
the summation of the products of overlapping points
between the template and signal when the template is
centred on the point of the signal currently being con-
volved. The learning dialogue would closely follow this
definition. The system would build up two new ATNs:
one for the overlap integral and one for the top level of
the convolution.
6 Implementation and examples
The system has been implemented in Domain-Lisp (a
variant of Common Lisp) and is running on an Apollo
D N 3000 workstation. It has been tested using twelve
channel electroencephalogram data collected by collabor-
ators of the Ophthalmic Optics Department at UMIST:
this data is called ‘the Berlin EEG data’. Any time series
data could have been utilised equally well. The test data
consists of files containing twelve independent series of
points of equal length. An example session will be
described.
The system is invoked by a user with the result that
the user’s file of ATNs is loaded. The user initialises the
system’s workspace by instructing it to ‘consider the
Berlin EEG data’. This results in the data contained in
the appropriate file being loaded into the current result
slot of the workspace. Suppose the system is requested to
‘display the samples between 0.5 and 1.5 milliseconds in
the first five channels’. This command is placed in the
current command slot and the earlier command trans-
ferred to the previous command slot. As a result of
376
parsing the current command the default time series
frame will be instantiated. Further parsing will result in
the appropriate sections of channels one to five being
selected and inserted into the data slot of the time series
frame and the result slot of the workspace. Routines
associated with the display frame autoscale the data as
required, generate labels for the display’s axes (the default
labels are ‘time’ on the horizontal and fiducial marks on
the vertical axes). At this stage the system’s graphical
output is as shown in Fig. 6. In the upper portion of the
screen are displayed the previous (left hand side) and
current (right hand side) outputs of signal processing
commands. The left hand side is blank at this stage as the
previous command did not result in any graphical
output. In the lower portion of the screen are displayed
the current and previous commands typed by the user.
The user may now give this data a name by instruct-
ing the system to ‘call this group 1’. The pronoun pro-
cessor associates the data in the current result slot with
‘this’ and binds it to the name. This command has no
graphical results, the current result display is therefore
empty, the previous result display now contains what
used to be the current result, i.e. the five time series of
data. The system also copies the result slot into another
portion of the workspace and labels this portion ‘group
1’. The system will produce an overwrite error message
informing the user that he has lost any previous data
with the same name.
If the user now types ‘display samples between 0.5 and
1.5 milliseconds in the last five channels and call it group
2’ a similar series of actions will be performed. The
system will now display the data in the top right portion
of the screen, the top left portion will be empty (Fig. 7).
The data will again be duplicated into the workspace and
labelled. The user may now use the names given to the
data in mathematical formulae: ‘display group 1 -
group 2’ would result in the difference between equivalent
points in the two data sets being displayed in the current
result window. The previous result window will now
display the original group 2 data. Had the user typed
‘group 1 - group 2’ the computation would have been
performed, but the result not displayed, Fig. 8.
Suppose now that the user types ‘remove the DC com-
ponent’. Parsing of this command will fail as remove and
DC component are unknown to the system. The system
will therefore enter its learning mode and prompt the
user to ‘clarify remove (x-1)’. The user may respond
with a suitable definition of this entity: ‘subtract x-1
from each point of the data’. The system will then seek
clarification of any further unknown phrases in the input,
subtract is a system primitive and requires no further
clarification, DC component is an unknown concept and
therefore requires clarification. The user will respond as
he thinks is appropriate, possibly by typing ‘divide the
sum of the data points by the number of data points’. It
is intended that as the user defines algorithms, the system
will build a tree like diagrammatic representation (c.f. the
Jackson diagram), this has not yet been implemented.
Having had the algorithm defined the system may
execute the command, resulting in the graphical output
of Fig. 9.
7 Conclusions and discussion
A restricted natural language interface for experts in the
domain of signal processing has been implemented. It
provides the experts with the facility to investigate the
effects of applying algorithms to their data without them
I E E P R O C E E D I N G S . Vol. 137. P t . E . N o . 5. S E P T E M B E R 1990
 TIME (MSECS)

E.E.C DATA

F I V E CHRNNELS

YOUR PREVIOUS INPUT => CONSIDER THE BERLIN EEG DATA

Fig. 6 Display o f a subset ofthe data

E.E.G DATA

l
DISPLAY THE SRMPLES BETUEEN 0.5 RND 1.5 MILLISECONDS I N THE IST

F I V E CHANNELS AND CALL IT GROUP 2

I/ YOUR PREVIOUS INPUT :> DISPLRY THE SRNPLES BETYEEN 0.5 AND 1 . 5

MILLISECONDS I N THE F I R S T F I V E CHRNNELS

THE PREVIOUS VALUE YOU HAVE GIVEN GROUP 2 WILL BE OVERWRITTEN

Fig. 7 Different subset ofthe data has been extracted and given the name 'group 2

IEE PROCEEDINGS, Vol. 137, P t . E, N o . 5 , SEPTEMBER 1990 311

.-* I

'I T I M E (MSECS) T I M E (MSECS)

E.E.G nnin
i E.E.t DRTA

I
I
DISPLfW GROUP 1 - GROUP 2

1 YOUR PREVIOUS INPUT => CALL THIS GROUP 2

I THE PREVIOUS VALUE YOU HAVE GIVEN GROUP 2 WILL BE OVERWRITTEN

REMOVE THE DC COMPONENT FROM CHANNEL 6 AND DISPLAY THE RESULT

YOUR PREVIOUS INPUT => REMOVE THE DC COMPONENT FROM CHANNEL 6

PRESENT ACTIVE COMPONENT => DISPLAY THE RESULT

~~ ~ ~ ~ ~ ~ _ _ _ _ _ _ _ _ _ _ _ _

Fig. 9 DC component, having been taught to the system, hap heen evtracted from each datum

318 IEE PROCEEDINGS. Vol 137. Pt. E. No 5 . S E P T E M B E R 1990

having to code the algorithm in a programming lan-
guage, nor learn an operating system. The interface is
intended to be used primarily by non programmers since
experienced programmers may not be tolerant of the
relatively slow execution of ‘programmes’ under this
system. Should useful algorithms be developed it is envis-
aged that they would be recoded procedurally. The
system may also be useful as a means of determining the
experts’ preferred means of expressing commands.
Augmented transition networks have been used to
represent the allowed forms of user input. It has been
shown that one network is able to specify most forms of
the same command. No claim is being made that the
ATN representation is complete, i.e. it is not able to rep-
resent all forms of a command. Indeed if the input cannot
be recognised as one of the allowed patterns then the user
will be prompted for a definition of his command.
The major novel aspect of the system is its learning
capability. When unknown patterns are encountered a
dialogue is initiated in which a definition for the
command is supplied. A new transition network will be
generated from this definition. A potential disadvantage
of this strategy is that it may cause a multitude of syn-
onymous networks to be produced if the user defines a
new command and then.calls it by different names. The
system would then request clarification of each new name
and would generate a new ATN for each definition. This
may be overcome by expanding all user definitions into a
tree-like form and then attempting to match nonleaf
nodes of pairs of trees according to the identity and ord-
ering of their subtrees. If matching nodes are discovered
the subtree of one may be deleted and that node made a
synonym of the other. Since this will be a time consuming
activity, it is not intended to be performed each time a
new command is defined, rather it would be done as a
background task after an interactive session had been
completed.
The smallest complete input to the system will consist
of a noun phrase and a verb phrase. The noun will
specify the data to be used by the action specified in the
verb phrase. The system is able to understand an input
sentence from which one of these phrases is omitted, pro-
vided that the missing phrases may be inferred from the
previous sentence.
The system has been designed, implemented, and
tested, but extensive user trials have not taken place.
Several volunteers with a range of computer experience
have attempted a number of simple programming tasks
using the system. None of these volunteers expressed dis-
satisfaction with the system.
A gradation of specificity may be identified in the
natural language interface. At the most general level the
transition network parser may be used in any applica-
IEE PROCEEDINGS. Vol. 137. P t . E. N o . 5. S E P T E M B E R 1990
tion, it could be thought of in the same way as one would
think of the interference engine of an expert system. The
transition networks that form part of the input to the
parser are less general (continuing the analogy they
would correspond to the expert system’s rules). The most
general of the ATNs are the metanetworks used to define
programming constructs, they could be used in any auto-
matic programming system. Secondly there is a group of
domain specific networks, the command networks, which
would find applications and any system performing
mathematical or signal processing operations. Finally the
user defined networks will probably be the least useful
outside this system, being specifically defined for this
application.
The noun phrases which describe the data to the
system are highly domain specific. If the system was to be
used as an interface to any other system then it is likely
that most or even all of the noun phrases would have to
be redefined. A means of specifying noun phrases to the
system is being investigated ; two teaching modes would
then be defined, one for teaching new action specifi-
cations and one for learning new data specifications. Ulti-
mately an interface could be produced which would
know a minimum of action and data specifications and
could be completely tailored by the user for a specific
application.
8 References
1 BALLARD, B.W.: ‘Semantic processing for a natural language
programming system’. PhD dissertation, Duke University, 1979
2 FINK, P.K., SIGMON, A.H., and BIERMANN, A.W.: ‘Computer
control via limited natural language’, IEEE Trans., 1985, SMC 15,
pp. 54-68
3 HEIDORN, G.E.: ‘Automatic programming through natural lan-
guage: a survey’, IBM J . Res. Develop., 1976,20, pp. 302-313
4 WOODS, W.A., KAPLAN, R.M., and NASH-WEBBER, B.: ‘The
lunar sciences natural language system: final report’. Bolt, Beranek
and Newrnan Inc, Cambridge, USA, 1972
5 PLATH, W.J.: ‘Request: a natural language question answering
system’, IBM J . Res. Develop., 1976,20, pp. 326335
6 WALTZ, D.L.: ‘An English language question answering system for
a large relational database’, Comm. A C M , 1978, 21, pp. 5 2 6 5 3 9
7 WARREN, D.H.D., and PEREIRA, F.C.N.: ‘An efficient, easily
adaptable system for interpreting natural language queries’, Am. J .
Computation Linguistics, 1982, 8, pp. 11&122
8 HENDRIX. G.G.:‘Human engineering for applied natural language
processing’, I J C A I , 1977,5, pp. 183-191
9 CHOMSKY, N. : ‘A transformational approach to syntax’, in
FODOR, J.A., and KATZ, J.J. (Eds): ‘The structure of language’
(Prentice Hall, Englewood Cliffs, 1964)
I O WOODS, W.A.: ‘Transition network grammars for natural lan-
guage analysis’, Comm. A C M , 1970,13, pp. 591-606
1 1 PETRICK, S.R.: ‘On natural language based computer systems’,
IBM J . Res. Develop., 1976, 20, pp. 314-325
12 DAHL, O.J., DIJKSTRA, E.W., and HOARE, C.A.R.: ‘Structured
Programming’ (Academic Press, London, 1972)
779