Novel approach of Domain Specific Ellipsis Handling in Question Answering Systems
Rahul Chitturi
Language Technology Research Center, IIIT-Hyderabad, INDIA
rahul_ch@students.iiit.net
Abstract
Human conversations often tend to be incomplete. Many a time, we tend
to shorten our conversations. The notion of omission from a text of one or
more words that are obviously understood, but that must be supplied, to
make a construction grammatically correct is called ellipsis [1]. In a con-
versation, the computer should be in a position to handle the ellipsis de-
pending on the context, previous dialogues and knowledge. Given a
specific domain question answering system, we deal with how to handle
ellipsis in that particular domain. In this paper we classify the ellipsis into
three types and try to provide solution for each of the three cases taking
an example of the Railway Domain. The evaluation of this algorithm is
done comparing the results with that of well known Question Answering
Systems, which proves that this approach is portable for domain specific
systems.
1 Introduction
The development of widespread computer technology has
changed many of our daily practices. Unfortunately, even today
the computers lack the very basic sense of naturalness in commu-
nicating with man. The creators of computer technology can
lessen the disruptive force of the technology by practicing good
design. Well designed computer systems should be useful, us-
able, easily learned, easily communicative and perform functions
that let people do the things they want to do. It is this fundamen-
tal necessity, which is ultimately leading the computer scientists
to overcome this barrier, concentrating on the natural means of
communication. Tremendous research is being carried on the
Natural Language Processing, Vision, etc now a day. The prob-
lem which we deal in this paper is the Ellipsis Handling in a
Natural Language Dialogue System.
Ellipsis structures pose a crucial problem for Natural Lan-
guage Processing systems, designed to provide text understand-
ing or to handle dialogues. They contain information which is not
overtly expressed, but which must be recovered through the iden-
tification of an antecedent or previous occurrences.
In a domain specific dialogue system, a machine answers queries
specific to that domain. For the dialogue to be as natural as pos-
sible, the system should be able to handle incomplete questions.
In order that the machine understands the query, the complete
query corresponding to an incomplete query has to be generated.
Let s see the example of ellipses in the railway reservation do-
main. The queries numbered are in the actual conversation and
their exact meanings are given correspondingly.
Example 1
Query 1: What is the next train to Calcutta?
Answer: Train number 1024.
Query 2: When does it start?
Exact query: When does the train 1024 to Calcutta start?
Query 3: Which platform?
Exact query: To which platform will the train 1024 arrive?
Query 4: When does it arrive in Bangalore?
Exact query: When does that train 1024 arrive in Bangalore?
Query 5: And Delhi?
Exact query: When does that train 1024 arrive in Delhi?
The problem which we deal in this paper is, given a conversation
as in example 1; the exact(Complete) queries should be obtained.
Complete queries are the queries for which the SQL queries can
be generated. This problem is first handled with the syntactic
cues from the preceding queries. If there is no much clue then
semantic cues are used to handle the situation which is not gener-
ally employed in the QA systems. The present QA systems like
AnswerBus [8], Quartz [9], Pai [10] don t take care of this ellip-
sis, which is quite essential in a natural conversation. Even the
popular systems like START use only the syntactic information
to handle the ellipsis [7]. In our paper, we present the semantic
approach which handles many of the complex ellipsis to make the
conversation more natural. This comparison is made in the
evaluation section (7).
2 Issues in handling ellipsis, in a question
answering system
2.1 Identifying the complete queries
Identifying the completeness of a given sentence is the very basic
issue in ellipsis handling. In the example 1, the first query is a
complete sentence and the rest are incomplete sentences. It is
quite complex to identify the complete sentence. Even if the sen-
tence structure is considered, for a given complete structure there
can be sentences that are not complete [2].
2.2 Scope of the context
Generally there is a perplexity regarding the number of queries
that should be kept in the memory, so that if they are referred to,
the required knowledge can be appropriately retained. It is diffi-
cult to retrieve the desired query from its elliptical notation in the
given knowledge base. This is clearly understood looking at the
example 1. In this example, in order to handle the ellipsis in
query 5, all the information from the first query is indispensable.
So, the problem here is how many previous queries should be
kept in the memory and also the way in which they should be
stored.
2.3 Entities in the domain
Generally, there is a mapping difficulty between the entities in
the Entity Relationship Diagram of a Database Management Sys-
tem and the entities in the domain that is being queried. It is
worthwhile to note that the entities in the DBMS are different
from the entities that are to be modeled semantically as in Dialog
Systems. This can be well understood from the discussions in the
later part of the paper.
The queries in a question answering system can be divided into
three types. This classification also depends on the type of do-
main and the type of queries that are going to be handled. Based
on the experience that is gained from the structure of the queries
in the Railway Reservation Domain, the generalization is done on
the following classification for all the question answering sys-
tems.
3 Difference between the ellipsis in discourse and
the question answering systems
3.1 Ellipsis in Discourse
The author of the reference [6] mentions that there are various
ways to describe the different types of ellipsis occurring in Eng-
lish and other languages]. Sanders (1977) uses alphabetic charac-
ters to identify the six different positions in which ellipsis can
occur, ranging from the first position in the first clause (position
A) to the last position in the second clause (position F):
ABC&DEF
Although there is disagreement about precisely which positions
permit ellipsis in English, most would agree that English allows
ellipsis in positions C, D, and E. Example (2) illustrates C-
Ellipsis: ellipsis of a constituent at the end of the first clause
(marked by brackets) that is identical to a constituent (placed in
italics) at the end of the second clause.
Example 2(C Type): The author wrote [ ] and the copy-editor
revised the introduction to the book.
Examples (3) and (4) illustrate D- and E-Ellipsis: ellipsis of, re-
spectively, the first and second parts of the second clause.
Example 3(D Type): The students completed their course work
and [ ] left for summer vacation.
Example 4(E Type): Sally likes fish, and her mother [ ] hamburg-
ers.
These types predominantly look at the intra-sentential ellipses.
3.2 Ellipsis in Question Answering System
As seen in Example 1, the ellipsis in the QA systems is very dif-
ferent from the general ellipsis. These are basically inter-
sentential ellipses. The case in Example 2 doesn t come into pic-
ture in QA systems. Also in the Examples 3 and 4, there is a lot
of structural difference from the Example1. The author of the
reference [6] mentions that 86% of the elliptical coordinations are
of type D. C accounts for 2% and E for 5.5%. So, the ellipsis in
the QA systems cannot be applied to the general ellipsis.
4 Classification of elliptical queries in a ques-
tion answering system
In this paper, we classify the ellipsis in a question answering
system into three types. The first two types have syntactic cues.
The third type is based on the semantic cues.
4.1 Type 1 (Preposition Based)
Though this seems to be very trivial for ellipsis han-
dling, most of the ellipsis in a domain can be handled by this.
This type of ellipsis is identified by the prepositions in the query
or sentence. This is easily understood by the following example:
Example 5
Query 1: Is there any train from X (station) to Y (station)?
Query 2: To S (station)? Or From T (station)?
4.2 Type 2 (Grouping Based)
Let s see the following example:
Example 6
Query 1: At what time will X (Train) arrive?
Query 2: What about Y(Train) ?
In this case there will not be any prepositions. So these can be
handled only by identifying the group of Noun Phrase (NP) to
which it belongs. One might get a doubt that how is this different
from the previous type (refer 3.1). Let us now see the following
example
Example 7
Query 1: When is the train from Bombay to Delhi?
Query 2: To Calcutta?
If we use grouping based method then we give no importance to
the preposition. This results in ambiguity that which should the
entity refer (Bombay? or Delhi?).
4.3 Type 3 (Semantic Based)
All those ellipsis which cannot be classified as the above two
types come under this type. For this type, a semantic diagram can
be built from the Entity Relationship diagram of the DBMS of
the given domain. This can be easily understood by looking at the
following diagrams. (Please refer to Fig.1 and Fig. 2).
Example 8
Query 1: When will the train X arrive?
Query 2: To which platform?
Every query can be handled by this type. But as this type is
related to semantics, this gives only basic semantic relations. The
first two types which are syntactically solvable are more accurate
in giving the exact relationship.
5 Algorithm for Ellipsis handling
In this paper the ellipsis handling problem is divided into four
parts. First the completeness of the queries is identified. Then the
entities in the query need to be mapped to that of the domain. The
queries along with their mapped entities are then analyzed. The
analyzed queries are kept in memory so that the ellipsis in subse-
quent queries can be handled.
5.1 Identifying the complete queries
The syntactic structure could be used with its corresponding se-
mantics, to obtain the semantics for the complete sentence. In this
case, the anaphoric expression is constrained to have the same
semantics as the complete expression [3]. But in our case, since
this is a domain specific system the queries in the domain are
limited. Finally, these have to be mapped to the DBMS queries.
So, a set of complete queries can be identified which are related
to that domain and for which the DBMS queries can be mapped.
These can be treated as complete queries. All these complete
queries are stored in the beginning. As simulating a human con-
versation is a very complex problem, some laborious work has to
be done in the initial stages of the system. This can be even
automated using speech recognition systems at the field of our
interest. To enact the human conversation a lot of data is required intervention these queries can be checked if they are complete.
for training the system. Using speech recognition systems the These can be used as templates for these complete queries.
queries in the domain can be obtained. And with little human
memory. So, in the next incomplete sentence if the same type of
preposition entity occurs then the previously entered value is
Num Name Seats
replaced by the present value.

Destination Source Generally, while speaking more stress is put on the head noun of
Train the sentence. So, the head noun of a complete dialog is identified.
Then whenever an incomplete dialog appears, the relationship
Time Day between the head noun of the previous complete query and the
s head noun of the incomplete dialog is identified. If in database,
there are many queries with only those two heads as entities, then
Plat-
they are returned. If no relationships exist between the two enti-
Travels Arrives
Name Dis ties then null is returned.
Book
s Example 12
Pas- Na Loca-
senger
When will the train X (Group: Train_specific) arrive?
PNR Station Train X is the head NP of the query
Ad-
dress
To which platform (Group: Platform) ?
Book- Platform is the head NP of the query
Counter ing
Offers
Id Then the relationship between the Train_specific group and the
Platform group is identified. Then all the queries with only these
Avails Con-
cession
two semantic entities are returned.

Example 10
Is there any train from X (place/station) to Y (place/station)?
Typ Percentage To Delhi? ;{ To Station_name} is together treated as the entity
e destination . Then to Y should be replaced with to Delhi
Figure 1 Entity Relationship Diagram for Railway Reservation
5.3.2 Group Based Ellipsis
System
The entities which are left after the processing the prepositions,
Example 9
will fall into some group. For example Delhi Express , Train
1) Will the {Train_specific} go from {Source} to {Destination}?
number 4567 , etc refer to a specific train. If an incomplete query
comes, then the value for that group in previous complete query
Train_specific is a specific train { Train number 2039 , Delhi
is replaced with the new value.
Express , etc} Source is a station or place { Delhi , Mumbai
station , etc} Destination is a station or place { Delhi , Mumbai
Example 11
station , etc}
At what time will X (Group: Train_specific) arrive?
What about Y (Group: Train_specific)? This Y is substituted
5.2 Matcher in the previous complete query in the place of X.

For each entity in the domain, all the possible values for that
entity are stored in the semantic graph. So, whenever a noun
phrase appears, it is matched with all the possible values of each
entity. Thus the noun phrases which are the entities in our domain
are identified. The entities need not be noun phrases but in this
paper we used only some defined set of noun phrases as the enti-
ties. The output of the matcher will be given to the ellipsis han-
dler.
5.3 Ellipsis Handler
The following methods have to be employed one after the other
in the order.
5.3.1 Preposition Based Ellipsis
The prepositions which are important in handling ellipsis in the
given domain are noted. Whenever these prepositions occur be-
fore a semantic entity, they can be treated as a separate preposi-
tion entity, which is different from the original entity and the
preposition. And the most recent value of this will be kept in the
5.3.3 Semantic Based Ellipsis
In the semantic graph, some entities have relations between one
another. The basic relationship between the possible semantic
entities should be kept in the database in the beginning.
These three types (3.1, 3.2, and 3.3) are not mutually exclusive.
But the procedure and the order in which they are applied is very
important. As shown in example 3, if the solution for the second
type is applied first, then there will be some problems. So, one
has to apply the solutions for these types one after the other. As
first two types are more accurate, first apply 4.1, then 4.2. If the
queries cannot be handled by these two types, then apply 4.3.
This approach would handle most of the ellipsis in that domain
5.4 Scope of the Context
It is very complex to know how many queries should be kept in
the memory. It depends on the type of domain. For example In-
teractive NLI agent [4] supports natural language queries and
commands along with a search history so that users can use their
queries based on the previous search results.
If the dialogues in the domain are kept in the memory, it becomes
very difficult to handle the queries. So, a hash of all the entities is
maintained. That is only entities are stored. At first, the entities
should be given the default values. If some other value is occurs
then the most recent value for that entity is stored.

R1
Pnr Address
Train type Pnr_number
Passenger name R9

Train R5
Specific Train Booking Counter
>Train name Counter id
>Train number R7
R8
R2 R4
R6
Platform Concession
Platform number Concession Type
Station
Source {To station}
Destination {From sta-
R3
tion}

Figure 2: Semantic Graph, Edges indicate Basic Relations between the semantic entities which are in ovals and their attributes which are in
rectangles. An example Basic Relation between Train and Platform: To which Platfrom will the train arrive?
`
In the example 1, the word train in query 1 is identified as entity
The Mechanism of Ellipsis Handling Train . Similarly Calcutta is identified as Destination
(Destination is intermediate station in which the train arrives). In
query 3, the word platform is identified as Platform . In query4,
Bangalore is identified as Destination , so Calcutta is replaced
Natural language query with Bangalore, as Calcutta is the latest value for entity
Destination . In query 5, the Delhi is identified as Destination ,
Defaults & and Bangalore is replaced with the Delhi, as Delhi is the latest
Possible values of value for entity Destination .
entities Entity Identifier
6. Snapshot of the output
Templates
The following is the exact dialog with the system
Complete query Identifier user: Tell me about the availability status of all train running
from secundrabad to Delhi on 03-03-05?

Incomplete query sys: ***Tell me about the availability status of all {train; Train}
Memory running {from secundrabad; Source} {to Delhi; Destination} on
Storage {03-03-05; Date}?
Ellipsis Classifier
Semantic sys: THIS IS ASSUMED AS A COMPLETE QUERY
Graph ------------------------------------------------------------------
user: to calcutta ?
Ellipsis Handler sys: ***{to calcutta;To_Station} ?

sys: DID YOU MEAN? Preposition based ellipsis

Ellipsis free query sys: Tell me about the availability status of all train running from
secundrabad to Calcutta on 03-03-05?
sys: ***Tell me about the availability status of all {train; Train}
Figure 3: Schematic Diagram of the mechanism of ellipsis running {from Secundrabad; Source} {to Calcutta; Destination}
handling on {03-03-05; Date}?
------------------------------------------------------------------
user: What about on 04-03-05?
sys: ***what about on {04-03-05; Date}?
sys: DID YOU MEAN? Grouping based ellipsis
sys: Tell me about the availability status of all train running from
secundrabad to Calcutta on 04-03-05?
sys: ***Tell me about the availability status of all {train; Train}
running {from secundrabad; Source} {to Calcutta; Destination}
on {04-03-05; Date}?
-----------------------------------------------------------------
user: To which platform?
sys: recognized the query as: ***To which {platform; Platform}?
sys: DID YOU MEAN? Semantic based ellipsis
/* Complete queries with {Train; Platform} Train is the head NP
of the latest complete query and the Platform is the head NP of
the previous query */
sys: to which platform will the train X (the name of the train
running from secundrabad to Calcutta on 04-03-05) arrive ?
sys: ***to which {platform; Platform} will the {train; Train}
arrive
-----------------------------------------------------------------
7. Evaluation
Many of the present QA systems do not handle the ellipsis. So,
the results of our algorithm are compared with that of the START
system which handles ellipsis. Also the START system handles
only the ellipsis considering the previous query in a dialogue as
the context [7]. Generally in human conversation this is not the
case. In our case the till the occurrence of complete query all the
incomplete queries are handled.
This approach is tested in three Domains - Railway Reservation
System, Course Registration System, and Library Maintenance
System. For each system we took 100 test cases such that all the
three type of ellipses are handled. The test cases are the dialogs in
that domain which have some inter-sentential ellipses as in ex-
ample (1). These are tested with our algorithm and START sys-
tem. Table 1-3 shows the result in all the three systems and types.

Table 1. Comparison with START system in Railway Reservation Domain

System Accuracy for Accuracy for Accuracy for
Railway Reservation Type 1 Ellipsis Type 2 Ellipsis Type 3 Ellipsis
Total Test Cases: 40 Total Test Cases: 35 Total Test Cases: 25
Algorithm discussed 100% 97.14% 80%
START system 57.5% 42.85% 0%

Table 2. Comparison with START system in Course Registration Domain

System Accuracy for Accuracy for Accuracy for
Course Registration Type 1 Ellipsis Type 2 Ellipsis Type 3 Ellipsis
Total Test Cases: 35 Total Test Cases: 30 Total Test Cases: 35
Algorithm discussed 100% 93.3% 65.71%
START system 54.2% 40% 0%

Table 3. Comparison with START system in Library Maintenance Domain

System Accuracy for Accuracy for Accuracy for
Library Maintenance Type 1 Ellipsis Type 2 Ellipsis Type 3 Ellipsis
Total Test Cases: 40 Total Test Cases: 30 Total Test Cases: 30
Algorithm discussed 100% 96.6% 83.3%
START system 52.5% 50% 0%

8. Future work 10. References

The endeavor from here onwards would be to identify ellipses 1. http://odur.let.rug.nl/~usa/LIT/chap10.htm

involving NPs, Verb groups and also for semantic nets. The idea
would be to make the system respond instantaneously to user
queries and use this query as a supervised input to generate a
database which relates to similar patters easily the next time they
are keyed in. Also a speech component identifying prosody ef-
fects is planned.
9. Acknowledgements
I would like to Dr. Dipti Misra Sharma and Prof. Rajeev Sangal,
LTRC, IIIT-Hyderabad who helped us a lot in this project, with
their feed back.
2. Mary Dalrymple, Stuart M. Shieber, and Fernando :
Pereira. 1991. Ellipsis and higher-order unification.
Linguistics and Philosophy, 14:399-452.
3. Andrew Kehler, Common Topics and Coherent Situa-
tions: Interpreting Ellipsis in the Context of Discourse
Inference In Proceedings of the 32nd Annual Confer-
ence of the Association for Computational Linguistics
(ACL-94), pp. 50- 57, Las Cruces, June, 1994.
4. Lee Jong-Hyeok, Rho Hyunchul, Park Young-Tack,
Choi Joongmin, Seo Jungyu Interactive NLI Agent
 For Multi-Agent Web Search Model - Geunbae, Jong-
Hyeok..(1998) nlp.postech.ac.kr/lab_papers/9808_iai
w_gblee.ps
5. Koeneman, Olaf, Sergio Baauw & Frank Wijnen
(1998). Reconstruction in VP-ellipsis: Reflexive vs.
non-reflexive predicates. Poster presented at the 11th
Annual CUNY Conference on Human Sentence
Processing. New Brunswick, NJ, March 19-21, 1998.
6. Charles F. Meyer, University of Massachusetts, Boston
: English Corpus Linguistics An Introduction, Series:
Studies in English Language
7. Boris Katz, MIT CSAIL: Discourse and Dialog in the
START Question Answering System, SIGDial 04
8. IR-244 2002 Pinto, D., Branstein, M., Coleman, R.,
King, M., Li, W., Wei, X. and Croft, W.B. QuASM: A
System for Question Answering Using Semi-Structured
Data , the JCDL 2002 Joint Conference on Digital
Libraries, pp. 46-55
9. David Ahn, Valentin Jijkoun, Gilad Mishne, Karin
Müller, Maarten de Rijke, and Stefan Schlobach (In-
formatics Institute, University of Amsterdam): A Re-
call Oriented Approach to Open Domain Question
Answering
10. http://www.di.unipi.it/~scordino/pai/pai.html
11. Jay Budzik and Kristian J. Hammond. Learning for
Question Answering and Text Classification: Integrat-
ing Knowledge-Based and Statistical Techniques.
AAAI Workshop on Text Classification. Menlo Park,
CA, 1998
12. Sanda Harabagiu, Marius Pasca, and Steven Maiorano.
Experiments with open-domain textual question
answering. COLING-2000. Association for
Computational Linguistics/Morgan Kaufmann, Aug
2000.
13. Sanda Harabagiu, Mihai Surdeanu, Rada Mihalcea,
Roxana Girju, Vasile Rus, Finley Lacatusu, Paul
Morarescu and Razvan Bunescu. Answering Complex,
List and Context Questions with LCC s Question- An-
swering Server. Tenth Text REtrieval Conference
(TREC-10). Gaithersburg, MD. November 13-16,
2001.