Deduction Engine Design for PNL-based

Question Answering System

Zengchang Qin1,2 , Marcus Thint2 and M.M. Sufyan Beg1,2

1
Berkeley Initiative in Soft Computing
Computer Science Division, EECS Department
University of California, Berkeley CA 94720, USA
{zqin, mmsbeg}@cs.berkeley.edu
2
Computational Intelligence Group
Intelligent Systems Research Centre
British Telecommunications (BT) Group
marcus.2.thint@bt.com

Abstract. In this paper, we present a methodology for designing a

Precisiated Natural Language (PNL) based deduction engine for auto-
mated Question Answering (QA) systems. QA is one type of informa-
tion retrieval system, and is regarded as the next advancement beyond
keyword-based search engines, as it requires deductive reasoning and use
of domain/background knowledge. PNL, as discussed by Zadeh, is one
representation of natural language based on constraint-centered seman-
tics, which is convenient for computing with words. We describe a hybrid
reasoning engine which supports a “multi-pipe” process flow to handle
PNL-based deduction as well as other natural language phrases that do
not match PNL protoforms. The resulting process flows in a nested form,
from the inner to the outer layers: (a) PNL-based reasoning where all
important concepts are pre-defined by fuzzy sets, (b) deduction-based
reasoning which enables responses drawn from generated/new knowl-
edge, and (c) key phrase based search when (a) and (b) are not possible.
The design allows for two levels of response accuracy improvement over
standard search, while retaining a minimum performance level of stan-
dard search capabilities.

1 Introduction
In general, a Question Answering (QA) system is a type of information retrieval
system triggered by an input query. In the particular context of this paper, QA
systems should produce direct answers/facts (rather than a ranked list of relevant
documents) for a question expressed in natural language. Accordingly, such QA
systems are regarded as the next generation search engines, since they require
capabilities for query analyzes, recognition of most relevant facts, and deducing
new facts and answers including the use of background or domain knowledge
where necessary.
Automated QA systems are broadly classified into two categories: open-
domain and restricted (or closed) domain systems. Restricted-domain QA deals
2 Qin et al.

with questions about a specific domain (e.g, telecommunications, medicine) or

a limited corpus, and is considered an easier task because natural language pro-
cessing (NLP) systems can exploit domain-specific knowledge frequently formal-
ized in ontologies. Open-domain question answering must deal with virtually
any topics, and commonly rely on the World Wide Web (Web) as a knowledge
source. Ask.com is an example of an early attempt at open-domain QA sys-
tem, and Google and MSN (Microsoft) are also integrating question-answering
capabilities in their search engines.
Even for restricted-domain QA systems, design complexity depends on the
type and size/scale of the knowledge source, input query format, and answer for-
mat requirements. QA system based on small, structured databases with struc-
tured queries that simply output a list of relevant facts are the least complex,
and those based on large, unstructured knowledge sources with natural lan-
guage queries that require formal/composed answers are the most complex. In
this paper, we discuss a problem of intermediate complexity, i.e. a QA system
based on small, unstructured knowledge sources with natural language queries,
where a list of the most relevant facts is an acceptable answer. We introduce the
application of PNL-based reasoning for restricted-domain QA systems, which
can increase the precision of answers for key topic areas. With a focus on well-
rounded system utility, we present a hybrid (“multi-pipe”) design that allows
integration of PNL-based reasoning, deductive reasoning with natural language
phrases, and default keyword based search as a minimal response output.
Although recent QA research investigates open-domain systems since the
Web is a vast and readily available knowledge source, restricted-domain QA
systems have also been studied. Some restricted-domain QA systems also rely
on data redundancy in the Web to validate answers or use the Web as an sup-
plemental knowledge source. For example, Chung et al. [4] studied a restricted
domain of weather forecasts for Korean cities where weather information is de-
scribed by well-defined HTML tags from a particular websites. Structured query
language (SQL) is used to query the facts of weather conditions, and the tem-
poral data normalizer converts temporal expressions such as today, this weekend
and now into absolute values that can be used in querying to the database. Tsur
et al. [11] presented a biographical QA system and empirical studies of machine
learning techniques (Ripper, a rule learning algorithm, and support vector ma-
chines (SVM) for biography classification). The system first attempts to answer
queries by searching a given collection of biographies, and if required informa-
tion is not found, attempts to find the answer on the Web. Ceusters et al. [3]
described a commercial QA system package for the health care domain based
on domain ontologies. It is a rule-based system where rules are derived from
expert knowledge. Benamara [2] presented the WEBCOOP system to generate
cooperative responses on the web, integrated with a knowledge base coded in
Prolog. Application example domain was restricted to tourism. Doan-Nguyen
et al. [5] discussed ways to improve precision of answers for restricted-domain
QA systems, through re-ranking of candidates proposed by generic Information
Retrieval (IR) engine, and heuristic improvements on the IR engine.
 Deduction Engine Design for PNL-based Question Answering System 3

2 Precisiated Natural Language

We investigate a novel application of Precisiated Natural Language (PNL) en-
hanced, restricted domain QA system. Vagueness and fuzziness is inherent in
natural language, and Zadeh [12] articulates that humans have a remarkable
capability to perform a wide variety of physical and mental tasks without any
measurements. Humans use perceptions which are intrinsically imprecise. Per-
ceptions partition objects into granules, with a granule being a clump of objects
drawn together by indistinguishability, similarity, proximity or function. Percep-
tions are both fuzzy and granular (referred to as f-granular), and they are difficult
for manipulation by computers. Zadeh proposed Precisiated Natural Language
(PNL) for reasoning with problems expressed in natural language. The new ap-
proaches are not necessarily intended to replace existing NLP tools, but offers
new tools for computing with f-granular information in natural language.
In PNL, the meaning of a proposition p, can be represented as a generalized
constraint on a variable. Schematically, this is represented as:
p→X isr R

In the generalized constraint expression X isr R, X is the constrained variable,

R is the constraining relation, and r is a discrete valued modal variable. The
“is” in isr is simply its natural meaning the conjugated verb “to be”. Thus,
the expression X isu R means X is usually R, and other defined modalities
include: possibilistic (r = blank); probabilistic (r = p); veristic (r = v); random
set (r = rs); fuzzy graph (r = f g); bimodal (r = bm); and Pawlak set (r = ps).
It is important to identify the constrained variable, which may also depend on
how the information will be consumed/interpreted. For example, a proposition
‘Mary is young’ can be represented in the two following forms:

p → Age(M ary) is young

and
p → P erson(young) isv M ary
which answer the questions “How old is Mary?” and “Who is young?” re-
spectively. In Zadeh [14], three basic operations on generalized constraints are
proposed (conjunction, projection, and propagation). If fuzzy sets are prede-
fined, these operations can be used to calculate the precisiated implications of
propositions.
A protoform (short for “prototypical form”) is a key concept that facilitates
deduction and reasoning with PNL expressions. Simple NL expressions with a
single “to be” verb phrase (in all conjugated forms, tenses, and modalities) can
be abstracted into respective “X is A” protoform, such as “Mary is young” or
“Price of a golf ball is two dollars”. Similarly, “John is a little older than Mary”
and “Price of a golf club is much more than the price of a golf ball” can be
abstracted to a “Y is X + B” protoform, and “Some balls are blue” can be
abstracted to a “Q1 As are Bs” protoform. Furthermore, if the first two phrases
are recognized as “age(Mary) is young” and “price(golf ball) is two dollars”, they
4 Qin et al.

can be abstracted to “f (X) is A” protoform. Details about PNL protoforms can

be found in [13] and [14].
Given that a subset of NL expressions are identified according to their proto-
forms deductions can be applied as follows. Given X is A, and Y is X + B it
can be reasoned that Y is A + B, and given Q1 A’s are B’s, Q2 (A’s&B’s) are
C’s, it can be computed that Q3 A’s are (B’s&C’s), where Q3 = Q1 • Q2 , where
• is a product in fuzzy arithmetic. In general, the following computational rules
apply to the protoforms:

X is A, (X, Y ) is B → Y is C, where µC (v) = maxu (µA (u) ∧ µB (u, v))

Q1 A’s are B’s, Q2 (A&B)’s are C’s → Q3 A’s are (B&C)’s, where (Q3 = Q1 • Q2 )
X is A → g(X) is B, where µB (v) = supu (µA (u)), v = g(u)
f (X) is A → g(X) is B, where µB (v) = supu (µB (f (u))), v = g(u)

Each of the forms above can be extended with respect to different modalities
(probabilistic, usuality, bimodal interpolation, fuzzy graph interpolation, etc.).
The deduction engine is designed to operate offline (i.e. without query input) as
well as to be query-driven [14].

3 Deduction Module
The deduction module is a major part of an overall PNL-based QA system.
A corpus of text documents are processed by the information extraction (IE)
module to identify and tag each sentence as a PNL protoform (thus far X is
A, Y is X + B, QAs are Bs, f (X) is A), causal fact (a causes b), if-then
fact, procedure, or simply ‘fact’ (none of the above). Similar techniques are
applied to the input question to identify the query-type (what, where, when,
how, how much/quantity). Details regarding the precisiation process and its
semi-automated implementation in the IE module are described in [1], and the
general system design and process flow of the QA system is discussed in [8].
Modules and processes related to the deduction engine are shown in Figure 1
below. The deduction engine has access to all corpus knowledge which were
precisiated by the IE module, and represented as a collection of fact-types.
If a sentence is a PNL protoform, it will be processed according to PNL rea-
soning. Else, phrase-based deductive reasoning is applied. In the presence of an
input query, our system analyzes the query key phrases and query-type, selects a
subset of relevant facts, and (where possible) combines with background knowl-
edge to generate the most relevant answer set according to a ranking system.
If deductive reasoning and concept matching (between query and facts) yield
results with low ranking values, a search engine (e.g., Lucene [6]) is invoked for
standard keyword-based search, which provides a minimal performance level.

3.1 Offline Reasoning

The deduction module is capable of operating in both online and offline mode. In
the offline mode, where there is no user input and thus no query, the deduction
 Deduction Engine Design for PNL-based Question Answering System 5

Fig. 1. The general structure of deduction engine for the PNL-based QA system.

module analyzes existing facts to see if new facts can be generated based on
PNL protoforms or phrase-based deductive reasoning. Thus far, standard logic
applied to phrases include: negation, transitivity, and chained reasoning.
– Negation: If we have If-Then rules or causal rules like “If A then B” or
“A causes B” (A → B), we can conclude that “if NOT B then NOT A”
(¬B → ¬A)
– Transitive Reasoning: If we have causal relations A → B and B → C, we
then can conclude that A → C.
– Chaining: chaining is a multi-step transitive reasoning. Given a set of causal
relations: A → B, B → C · · · E → F , we can conclude that A → F .
Semantic interpretation of chained reasoning results may sometimes appear
to be unclear or illogical, hence the deduction module supports an option to
show (explain) all intermediate results.
In addition to standard reasoning above, we also combine X is A facts with
related simple facts. For example, given the facts: (1) Carnivores eat meat. (2)
Lions are carnivores. The second fact is in X is A form but the first one is simply
a fact. The two noun subjects (or subject phrases) are treated as equivalent, and
via substitution of ‘carnivores’ by ‘lions’ in the first sentence, the deduction
module will conclude that “lions eat meat”.
As explained in [8] an application manager may provide background knowl-
edge in the forms of simple facts or abstract facts. During offline reasoning,
the QA system can also generate new knowledge by combing the facts from
a corpus content and background knowledge. For example, in a domain about
telecommunication, the background abstract facts: “wireless *yy is less secure
than wired *yy” and “wireless *zz simplifies physical installation” may be pro-
vided. The notation ‘∗yy’ and ‘∗zz’ are reserved variables, and the deduction
module searches for matches of phrase pattern surrounding the reserved words
6 Qin et al.

in the corpus text. For example, if another sentence contains information about
‘wireless router’, then new sentences “wireless router is less secure than wired
router” and “wireless router simplifies physical installation” are generated al-
though they do not explicitly appear in the corpus text.
We note that such knowledge augmentation may also generate some mean-
ingless sentences - if a phrase “wireless chat session” is found elsewhere, gen-
erated sentence “wireless chat session simplifies physical installation” does not
make sense. However, the generated knowledge are passively stored, and when
a user inputs a query, the online processing helps to filter sensible results. It
is more likely that sentences more relevant to the query will be ranked higher.
For instance, if asked, “what type of routers simplify physical installation?” The
correct answer “wireless router simplifies physical installation” will be ranked
higher than the phrase about wireless chat session.
Offline reasoning improves performance and execution time during online
use. The module operates at the phrase-level; each sentence in the corpus is
segmented into subject phrase, verb phrase, and object phrase, such that the
phrase components can be identified with the X, Y or A, B, C ... components
of the PNL or non-PNL protoforms above. Accurate recognition of matching
components (e.g. X in one subject phrase with X in another object phrase)
depends on the quality of the concept matching module for two input strings.
This module removes stop words and checks synonyms of remaining keywords
(stemmed & non-stemmed) using WordNet [7] to compare phrases at the concept
(rather than string) level. Given a query, the answer generation process flows as
described below.

3.2 Query-based Reasoning

The query is analyzed to extract its key phrases and query type (e.g. what,
where, when, how, how much). The latter is non-trivial, and requires analysis
beyond key word spotting, since “what is the cause...”, “what is the method...”,
“on what occasion...”, are actually “why”, “how”, and “when” type questions
respectively, despite the prominent “what” keyword. Similarly, other cases have
exceptions, and only cases with higher confidence (e.g. where, whereabouts, loca-
tion as indicators for ‘where’-type questions) are marked. Additional discussion
about query typing can be found in [1]. Since this query-typing is used as a
secondary factor to refine answer ranking, it is non-critical but helpful when
correctly typed, as further explained below.
From the corpus knowledge (comprising facts extracted from a specified cor-
pus), most relevant subset of facts are identified via concept matching with query
keywords. If the relevant facts are known PNL protoforms, they are passed to
the PNL-based deduction “pipe” and the deduction engine checks for associated
fuzzy set definitions. For example, a QA application for an auto sales department
may have fuzzy sets defined for “low”, “moderate”, “powerful”, “expensive”, “a
little more”, “a lot more”, etc. If fuzzy sets are defined, and the answer to a
query is in the form Y is A + B (e.g. “Horsepower of sports model is a lot more
than the horsepower of basic model”), where “Horsepower of basic model is low”
 Deduction Engine Design for PNL-based Question Answering System 7

our QA system will actually add the fuzzy sets for “low” plus “a lot more” and
produce the approximate defuzzified value. If the fuzzy sets are not defined, the
answer remains in the NL expression as “Horsepower of sports model is low
PLUS a lot more” (i.e. a lot more than “low”) which is just a linguistic interpre-
tation of Y is A+B form. In short, pre-defined fuzzy concepts enable computing
more precise responses in the PNL paradigm; otherwise the system will return
a linguistic interpretation of PNL protoforms.
As explained in [1] some “f(X)” protoforms (e.g. cost(a flat screen monitor))
can be identified from phrases like “A flat screen monitor is expensive”, and the
deduction module supports associated computations. Given the following facts:

– A mouse is cheap.
– A computer monitor is two hundred US dollars.
– A keyboard is fifty US dollars.

If asked “what is the cost of a computer monitor plus a keyboard?” the system
returns:

This is a ‘compound’ type question (e.g. f (X) plus f (Y )). Reasoned

from sentence: ‘A computer monitor is two hundred US dollars.’ AND
‘A keyboard is fifty US dollars.’ We can obtain:
—————————————–
cost(A computer monitor) PLUS cost(A keyboard) IS (two hundred US
dollars)+(fifty US dollars)

If we ask the cost for a mouse plus a keyboard, the answer will be ‘fifty US
dollars’ PLUS ‘cheap”. The defuzzification value will be returned if ‘cheap’ is
predefined by a fuzzy set. Otherwise, only the linguistic answers as above.

3.3 Ranking of Answers

If the relevant facts (to a query) are not PNL protoforms, they could be one of
the following (as tagged by the IE module): causal fact, if-then fact, procedure, or
just “fact”. These facts are processed through the phrase-based deduction pipe,
and the deduction engine (i) generates primary rankings of facts based on the
degree of concept match with the query, (ii) attempts deductive reasoning based
on new constraints supplied in the query, and (iii) generates secondary ranking
based on the query type (if applicable). Deductive reasoning with new con-
straints include negation, transitive and chained reasoning as explained above,
to produce an answer. In many cases, multiple facts are determined to be rel-
evant to a query, as they include the same concepts/key phrases appearing in
the query. Thus, during the secondary ranking process, ranks of causal or if-then
facts are incremented for ‘why’-type question, ranks of procedure facts would be
incremented for ‘how’-type questions, and facts containing quantity terms (nu-
meric or fuzzy terms such as “few” or “most”) are incremented for how-much
type questions. Our system displays N top ranked answers, where N may be
specified by the application manager.
8 Qin et al.

A basic ranking algorithm is the count of matched concepts between cor-

pus facts and user query. However, by offline reasoning, we can conclude some
implicit relations between sentences and use it to re-rank the sentences. For ex-
ample, consider the sentences:

1. Up to 16 telephones can be connected to one controller.

2. Up to 6 telephones can be connected to module-Alpha on the wall.
3. The controller is the heart of Beta.

and the query “how many telephones can be connected to Beta?”. Typical
keyword based systems will rank (1) or (2) as the highest answer, but our system
detects that (3) is a X is A form and “controller” and ”the heart of Beta”
is equivalent. Therefore, a new generated fact: “Up to 16 telephones can be
connected to the heart of Beta” is the highest ranked answer. In some cases
where the relevant subset is sparse or the query is ill-formed, and deduction
process can not be applied. If concept match results yield low rankings then the
highest scores of a standard search engine are returned.

4 More Examples
Additional details and output samples below summarize key functionalities of
the deduction module. Given example facts:

(i) Horsepower of the basic model is about 150.

(ii) Horsepower of the sports model is a lot more than horsepower of the basic model.

and key concepts defined by fuzzy sets: ‘about 150’ is defined by a triangu-
lar fuzzy set centered on 150 with width of the fuzzy set ± 10% (i.e. ± 15 for
this example). The term ‘a lot more’ is defined by a percentage (e.g. 100%) on
the range of the reference fuzzy set ‘about 150’. Since the two facts are PNL
protoforms X is A and Y is X + B respectively, the deduction engine produces
the result Y is A + B, and the following fuzzy addition is performed:
T riF uzzy(150, 15) + 150 × 100% = T riF uzzy(300, 15)

where T riF uzzy(a, b) represents a triangular fuzzy set whose center is at a and
the width is b. The final answer is returned as “about 300” where 300 is the
defuzzified value of the composite fuzzy set. QA system built on standard search
technology cannot provide this type of response. In practice, all key concepts in
a corpus are not likely to be pre-defined with fuzzy set representation, but for a
particular sub-section(s) where the application manager expects many questions
and desires to provide detailed answers, PNL-based computing offers value-added
performance.
Examples of phrase-based deductive reasoning are provided below. For ex-
ample, given two facts:
 Deduction Engine Design for PNL-based Question Answering System 9

(i) If over 50 percent of customers complain about new product WRTG54, all types of
WRTG routers will be recalled.
(ii) If all types of WRTG routers are recalled, we will start new sale on TRG100 type
of router.

If the query is: ‘Why did you start new sale of TRG100?’ The system returns
two answers:

(1) If all types of WRTG routers are recalled , we will start new sale on TRG100
type of router.
(2) If over 50 percent of customers complain about new product WRTG54 then we will
start new sale on TRG100 type of router.

If the query is: ‘Why would you not start new sale of TRG100?’. The answers are:

(3) NOT (we will start on a new sale on TRG100 type of router) IMPLIES NOT
(all types of WRTG routers are recalled).
(4) NOT (we will start on a new sale on TRG100 type of router) IMPLIES NOT (over
50 percent customers complain about new product WRTG54.
The second set of answers (3 and 4) result from the transitive implications of
facts (i) and (ii).

At this juncture, a result processor has not been implemented to ‘re-phrase’

answers in human-friendly form, but the reasoning logic is correct.

5 Summary

We presented an application of PNL-based reasoning, and in particular, dis-

cussed its role in a deduction engine for QA systems. While many NL expression
can be precisiated manually (i.e. with human interpretations of semantics), a
smaller subset is amenable to automated precisiation. As a first attempt at
(semi-)automated application of PNL-based reasoning, the input was limited
to simple NL sentences (i.e. with single verb phrase) that can be precisiated
and abstracted to PNL protoforms. PNL-based processing converts the complex
problem of NLP operations and deduction with natural language phrases into
simple substitution operations of algebraic expressions. Moreover, if associated
fuzzy concepts are pre-defined with fuzzy sets, more accurate answers (within
the constraints of f-granularity) can be computed. The limitation (in the context
of advanced search or QA applications) however, is that since only a subset of
natural language sentence forms can be precisiated in a (semi-)automated sense,
PNL-based computing cannot be used as a stand-alone solution, and must be
supplemented by standard NLP and information retrieval techniques to realize
a complete and practical system. The hybrid, multi-pipe approach described
above provides one way to implement an integrated solution. This system re-
10 Qin et al.

mains under development, and further development and testing are planned
using TREC’s QA data sources [9]. Further research topics also include concept
matching, ranking algorithms, different knowledge representation schemes such
as RDF and OWL formats [10], and integration of additional NLP tools such as
entity extractors for analyzing corpus knowledge and query phrases.

Acknowledgements

Qin and Beg are British Telecommunications (BT) Research Fellows in BISC
Group. This research was funded by BT/BISC Fellowship.

References
1. M. M. S. Beg, M. Thint, Z. Qin, Precisiating natural language for a question an-
swering system, submitted to the 11th World Multi Conf. on Systemics, Cybernetics,
and Informatics, 2007.
2. F. Benamara, Cooperative question answering in restricted domains: the WEB-
COOP experiment, Proceedings ACL 2004 Workshop on Question Answering in Re-
stricted Domains, 2004.
3. W. Ceusters W, B. Smith, M. Van Mol, Using ontology in query answering systems:
scenarios, requirements and challenges. In Bernardi R, Moortgat M (Eds.) Proceed-
ings of the 2nd CoLogNET-ElsNET Symposium, Amsterdam, pp.5-15, 2003.
4. H. Chung, Y.-I. Song, K.-S. Han, D.-S. Yoon, J.-Y. Lee, H.-C. Rim and S. -H. Kim,
A practical QA system in restricted domains, 2004.
5. H. Doan-Nguyen, L. Kosseim, The problem of precision in restricted-domain
question-answering: some proposed methods of improvements, Proceedings ACL 2004
Workshop on Question Answering in Restricted Domains, 2004.
6. O. Gospodnetic, E. Hatcher, Lucene in Action, Manning, 2004.
7. G. Miller, Wordnet: a lexical database, Communications of the ACM, 38(11), 1995,
pp. 39-41.
8. M. Thint, M. M. S. Beg and Z. Qin, PNL-enhanced restricted domain question
answering system, submitted to IEEE-FUZZ, London, UK.
9. Available: http://trec.nist.gov/data/qa.html
10. Available: http://www.w3.org.org/TR/owl-ref
11. O. Tsur, M. de Rijke, and K. Sima’an, BioGrapher: biography questions as a re-
stricted domain question answering task, Proceedings ACL 2004 Workshop on Ques-
tion Answering in Restricted Domains, 2004.
12. L. A. Zadeh, A new direction in AI - toward a computational theory of perceptions,
A.I. Magazine, Spring 2001.
13. L. A. Zadeh, From computing with numbers to computing with words − from
manipulation of measurements to manipulation of perceptions, Int. J. Appl. Math.
Comput. Sci., Vol. 12/3: pp. 307-324, 2001.
14. L. A. Zadeh, Toward a generalized theory of uncertainty (GTU) − an outline,
Information Sciences, Vol. 172: pp. 1-40, 2005