You are on page 1of 6

Contract Statements Knowledge Service for Chatbots

Boris Ruf1 , Matteo Sammarco1 , Marcin Detyniecki1,2,3

Abstract— Towards conversational agents that are capable


Chatbot client Chatbot framework Knowledge service
of handling more complex questions on contractual conditions,
formalizing contract statements in a machine readable way is
crucial. However, constructing a formal model which captures Ask question
the full scope of a contract proves difficult due to the overall 1. Intent classification
(knowledge service selection)
complexity its set of rules represent. Instead, this paper presents 2. Named-entity recognition
(parameter value extraction)
a top-down approach to the problem. After identifying the most
HTTP GET(query)
relevant contract statements, we model their underlying rules
arXiv:1910.04424v1 [cs.AI] 10 Oct 2019

HTTP 422 – Missing parameter


in a novel knowledge engineering method. A user-friendly tool Request parameter value
we developed for this purpose allows to do so easily and at Send parameter value
3. Update parameter-value pair in query
scale. Then, we expose the statements as service so they can
get smoothly integrated in any chatbot framework. HTTP GET(query)

HTTP 200 – OK

I. I NTRODUCTION Send response

For a long time, researchers in artificial intelligence (AI)


have been intrigued by the idea of developing a conversa- Fig. 1: A sequence diagram of a sample conversation flow.
tional agent that is capable of having a coherent conversation The knowledge service incorporates the contract statement.
with humans [1]–[3]. Recent breakthroughs in semantics and
speech recognition have given rise to hopes for robust solu-
tions to the problem [4], [5]. Major information technology
companies have released digital assistants and chatbot frame- However, it is extremely hard to apply this concept on real-
works to facilitate the building of conversational agents [6], world contracts based on existing contract law provisions
[7]. because smart contracts are “indifferent to the fundamental
However, research studies on the user perception and legal principles, such as lawfulness, fairness, and protection
expectations from users of chatbots indicate that the systems of the weaker party” [12].
still require significant improvements in order to provide Matching inquiries around a contract using a conventional
a meaningful experience [8]. Also, demand analysis iden- FAQ bot would be an option, in particular when using a deep
tified a need for specialized digital assistants in customer learning neural model for answer selection [13]. However,
facing processes, in particular in the insurance sector [9]. the use of static answers can quickly become complex and
A promising approach to advance in this field are compre- hard to maintain due to the arbitrary number of combinations.
hensive knowledge engineering methodologies which back In this paper, we propose a knowledge engineering
the chatbot and upgrade its conversational capabilities from methodology to model contract statements which support the
small talk to domain expert [10]. automated handling of such questions. We choose a top-down
In our case, we are interested in the domain of contract approach, manually selecting the most relevant statements
statements of insurance policies. In order to narrow down of a contract and modeling their underlying rules in a
the problem, we focus on the kind of questions that can straightforward and accessible manner. The rules are based
be answered based on linear rules which take a finite set on combinations of different parameter spaces. This allows
of parameter-value pairs as input. We assume that most for the modelling of complex statements while facilitating
customer support inquiries fall in the domain of our interest, easy maintainability. Conversational agents can query the
i.e. questions around coverage and pricing in real-world rules via API which allows for seamless integration with any
insurance policies. chatbot framework. We also assess the theoretical statement
Modeling contracts bottom-up as self-executing contracts, expressivity of our approach and demonstrate that it holds a
also known as “smart contracts”, is appealing due to its large significant potential of complexity reduction compared with
potential of automation and its resilience to tampering [11]. conventional FAQ bots.
The remainder of this paper proceeds as follows. We begin
1 Boris Ruf and Matteo Sammarco are with AXA, Paris, France with an overview of the overall architecture we propose and
{boris.ruf,matteo.sammarco}@axa.com
2 Marcin Detyniecki is with AXA, Paris, France and with Sorbonne introduce a toy scenario. Next, in Section III, we define the
Université, CNRS, LIP6, Paris, France and with Polish Academy of Science, expected input to the system, describe the theory behind
IBS PAN, Warsaw, Poland marcin.detyniecki@axa.com our concept of contract statement modeling, and outline the
©2019 IEEE different possible query scenarios. Then, Section IV studies
tennis p1 v1 v2 … vi
hiking
scuba diving basketball
p1=sport climbing
soccer
K1 K2 K3
p2 Vi+1 Vi+2 … vj
Vparameters
e1 e2 e3 eq


Turkey
p2=country Switzerland Nepal
Greece

K4 K5 K6 pm Vk+1 Vk+2 … vl

v7 v8 v9 Vl+1 Vl+2 … Vn Vresponses

r1=20 EUR r2=30 EUR r3=50 EUR r1 r2 rn-m

Fig. 2: Toy scenario: The hypergraph represents a contract Fig. 3: The schematic representation of a hypergraph H
statement of a fictional accident insurance statement. which models a contract statement. The vertices vi represent
the different sets of parameter values and also the possible
responses. Rules are defined by hyperedges ei which connect
several vertices.
the theoretical complexity reduction of our approach with
respect to the maximum and the actual number of questions
covered. Further, we provide the technical specification of
our prototype which realises the present concept in Sec- malized as rules of a contract statement. The properties of the
tion V. Finally, we conclude by providing insights in possible knowledge service are described in full length in Section III.
future fields of research.
D. Toy scenario
II. G ENERAL ARCHITECTURE
The proposed architecture to respond to customer support Throughout this paper we will illustrate all aspects of our
inquiries consists of a chatbot client, a chatbot framework proposal with a simple toy scenario. Its model is shown in
and several knowledge services. The conversation flow be- Figure 2. The subject is a fictional accident insurance which
tween those components has been illustrated in a sequence covers sports accidents in different geographical regions.
diagram in Figure 1. Prices vary depending on the kind of sport and the country.
We would like to automate responses to price requests based
A. Chatbot client on the customer’s parameters.
The chatbot client is the messaging interface provided to
interact with the customer support. It allows to send and III. K NOWLEDGE SERVICE
receive short messages and preserves the conversation in a The knowledge service plays the role of the domain expert.
local message log. It processes the query, identifies the matching rule and
B. Chatbot framework delivers the appropriate answer. In the following, we describe
the expected input to the service, the modeling of a contract
The chatbot framework manages the conversation flow statement, and the different query scenarios.
and preprocesses the inquiries. In this paper, we focus
on designing the knowledge service, the operations of the
A. Input query
chatbot framework are not in the scope of this paper. Thus,
we only briefly describe them as follows. Every inquiry The knowledge service expects a structured request con-
sent to the chatbot framework is preprocessed using Natural taining a set of query parameters as input. As previously
Language Processing (NLP) methods. First, a text classifier described, this request is prepared by the chatbot framework.
is used for intent classification. By understanding the matter In the context of our toy scenario, the customer could
of the inquiry, the framework can identify the corresponding for example enter the question “How much is an accident
knowledge service. Then, named-entity recognition is run insurance for scuba diving in Turkey?” in the chatbot client.
on the user message to retrieve the parameter-value pairs. The chatbot framework classifies the intent of the mes-
Finally, the parameter-value pairs are dispatched to the sage and match the corresponding knowledge service which
knowledge service as query parameters. handles pricing questions. Further, it identifies the required
parameters for this service, here sport and country. Using
C. Knowledge service named-entity recognition, the relevant parameter values, in
Every knowledge service represents an endpoint for one this example “scuba diving” and “Turkey”, are extracted and
specific type of question. It contains expert knowledge for- dispatched as query parameters to the knowledge service.
B. Contract statement Algorithm 1 Outline rule matching algorithm
We first propose to select relevant statements from a Input: H(Vparameters + Vresponses , E)
contract, that is, specific rules which are frequently subject x ∈ {(p1 , k1 ), (p2 , k2 ), ..., (pn , kn )} where pi ∈ P, ki ∈ S
of customer inquiries. We suggest to model such statements
as hypergraphs. The vertices of a hypergraph represent the Output: response, http code
different parameter values and also the possible responses of if ki valid then
each statement. Rules are defined by edges which connect Vmatches ← Identify all vertices vi ∈ Vparameters
several vertices. which are labeled with pi and include ki in the associ-
In our toy scenario, the statement is about pricing. The ated set of keywords
vertices represent the possible parameter values, here groups edge ← Identify edge ei ⊆ Vmatches , ei ∈ E
of sports and countries. They also stand for the responses, in if edge exists then
this example the different prices. The rules define the prices vr := edge ⊆ Vresponses
for each available combination of parameter values. response := label(vr )
We choose a hypergraph to model a statement because, http code := 200
unlike a simple graph, a hypergraph has edges which can else
join any number of vertices. Let H = (V, E) be a finite, if n < |P | then
undirected hypergraph with vertices V = {v1 , v2 , ..., vn }. response := pn+1
E is a set of hyperedges {e1 , e2 , ..., eq }, such that each http code := 422
hyperedge contains an arbitrary set of vertices, ei ⊆ V . More else
precisely, E is a subset of P(V ), where P(V ) is the power response := f alse
set of V . http code := 200
An schematic presentation of a hypergraph H as defined end if
above can be found in Figure 3. An instance of this hyper- end if
graph representing the statement of the toy scenario is shown else
in Figure 2. response := f alse
1) Vertices: The vertices are split in 2 different types: the http code := 400
ones that represent the parameter values, and the ones that end if
stand for the available responses.
Let vertices {v1 , v2 , ..., vl }, 1 ≤ l ≤ n be the first subset
Vparameters of V . They are each labeled with a string For example, in our toy scenario displayed in Figure 2, the
denoted as parameters P = {p1 , ..., pm } where pi ∈ S. S is rules show that the prime for “hiking” in “Switzerland”
the set of strings which consist of at least 2 characters. In would be cheaper than for “hiking” in “N epal”. On the
our toy scenario, the parameters are sport and country. other hand, insurance for “scuba diving” in “N epal” is
The labels are unique, pi 6= pj , and the maximum number not offered.
of parameters is the total number of vertices, thus 1 ≤ m ≤
n. Each vertex of Vparameters represents a set of possible C. Query scenarios
values for the given parameter. We describe those values as The necessary steps to find the right response to an
set of keywords Ki = {k1 , k2 , ..., kni } which consist of at inquiry have been outlined in the form of pseudo code in
least one string, thus where kj ∈ S, ni ∈ N. The keywords Algorithm 1. First, we identify all vertices of Vparameters
are required to be unique per parameter, thus kx 6= ky which match the parameter-value pairs provided in the query.
for each vi ∈ Vparameters where label(vi ) = label(vk ). Then, we search for a hyperedge that connects those vertices.
Otherwise, several rules could apply simultaneously, and For this purpose, we consider the following scenarios.
there would be no unambiguous result. In our continuous 1) Complete input query (default): If parameter values
example, the keywords are K1 = {“hiking”, “climbing”}, were provided for all parameters, and a fully connected
K2 = {“scuba diving”}, etc. hyperedge exists, we single out the vertex which belongs
Let vertices {vl+1 , vl+2 , ..., vn } be the second sub- to the subset Vresponses . Notice that by definition each
set Vresponses of V , labeled with responses R = hyperedge must include exactly one such vertex. As response
{r1 , r2 , ..., rn−m } where ri ∈ S. Responses are to be unique we send the label of this vertex to the chatbot framework.
in our model, thus ri 6= rj . In the toy scenario, the responses If the matched vertices are not fully connected by a
are R = {“20 EU R”, “30 EU R”, “50 EU R”}. hyperedge, it means that none of the defined rules in the
2) Edges: Edges represent the statement’s rules. We re- statement apply. Thus, the response is the boolean value
quire the rules to take at most one value per parameter, which f alse. Depending on the use case, this response may be
is why we only allow edges to connect vertices of different interpreted as “no”, or as “undefined” – impossible to make
parameters. Formally, this means that each hyperedge ei ∈ a valid statement.
E must contain vertices vx and vy so that label(vx ) 6= 2) Incomplete input query: If the query is incomplete,
label(vy ). Also, it must include a minimum of 2 vertices, a fully connected hyperedge may still exist, since we do
including 1 vertex of Vresponses , so that 2 ≤ |ei | ≤ |P | + 1. not require hyperedges to include vertices of every single
question-answer pairs, which is time-consuming to achieve
Statement expressivity
and difficult to maintain. On the contrary, the contract
5000
#v=2
#v=3
model presented in this paper induces that each statement
#v=4 represented by a hypergraph factorizes all possible answers
4000
into rules and as such serves as dynamic answer to multiple
questions.
3000
A. Statement expressivity
z

To estimate the maximum number of different questions


2000
that could be covered by a hypergraph, we need to add up all
possible parameter-value pair combinations for any possible
1000
hyperedge. To achieve this, we first count the keywords
represented by each vertex of Vparameters labeled with the
0
0 2 4 6 8 10
same parameter pi ∈ P and assign those numbers to a set σ.
#k Then, we can compute with the following formula the value
of z which represents the maximum number of conventional,
Fig. 4: Statement expressivity scenarios for a differing num- static question-answer pairs the present statement replaces:
ber of vertices per parameter (#v) over an increasing number
of keywords per vertex (#k). The number of parameters |P | z=
X Y
c (1)
and the number of response vertices |R| are fixed to 2 and
s∈P(σ) c∈s
3 respectively for better comparability.
where P(σ) is the power set of σ, and c is the element of
each subset of σ.
We observe that the size of |Ki |, which corresponds to
parameter. In this case, we send the label of the vertex which the number of values accepted for one parameter vertex vi ,
belongs to the subset Vresponses , just as described above. as well as the total number of parameter layers |P |, have a
However, if the matched vertices are not represented by a strong influence on the overall complexity.
fully connected hyperedge, the knowledge service prompts For the toy scenario, the value of z is calculated as follows:
the user for the missing parameter values until either a
hyperedge is found or the query is complete and contains σ ={(|K1 | + |K2 | + |K3 |), (|K4 | + |K5 | + |K6 |)}
values for all parameters. More specifically, the label of the ={(2 + 1 + 3), (1 + 2 + 1)}
first parameter pi ∈ P without value ki is sent to the chatbot ={6, 4}
framework with HTTP status code 422 (“Unprocessable
Entity”). The framework requests the parameter value from P(σ) ={{6}, {4}, {6, 4}}
the user, and finally resubmits the updated parameter query. z =6 + 4 + 6 · 4
This approach enhances the knowledge service with a very =34
basic capacity to manage part of the conversational flow.
As result, up to 34 conventional question-answer pairs can
For example, if the question “How much is an accident
be mapped to this single hypergraph with 3 levels and 9
insurance for scuba diving?” was asked in the toy scenario,
nodes.
the knowledge service would respond with “country” and
HTTP code 422, prior to providing a final response. The B. Total questions covered by a specific statement
chatbot framework could rephrase the request to “In which
In order to calculate the number of standard FAQ bot
country would you like to exercise the sport?”, include the
questions covered by a single given hypergraph, we sum up
answer of the user to the query and pass the completed
all possible parameter-value pair combinations for each hy-
parameter-value pair to the knowledge service.
peredge present in the graph. To achieve this, we first identify
3) Invalid input query: A parameter query may include
all vertices vi ∈ Vparameters included in a hyperedge. Then,
unexpected input, i.e. multiple values for the same parameter.
we count the keywords represented by each of those vertices
For the sake of simplicity, the knowledge service currently
and multiply the results. In order to compute t, the total
only detects such exceptions and leaves it to the chatbot
number of questions covered by the hypergraph, we repeat
framework to handle them appropriately. More concretely,
this procedure for all remaining hyperedges, and add up the
the knowledge service responds with the boolean value f alse
sums. The resulting formula is:
and the HTTP status code 400 (“Bad Request”).
X Y
IV. C OMPLEXITY REDUCTION t= |Kindex(v) | (2)
e∈E v=e⊆Vp
In theory, the previously presented type of questions could
be answered by conventional FAQ bots, too. However, such where E is the set of hyperedges, and Vp = Vparameters .
frameworks would require to write one explicit answer for For example, in the case of the toy scenario illustrated in
every single question. This would result in a very large set of Figure 2, the value of t is computed as follows:
(a) Interactive hypergraph: The vertices are displayed as boxes. The 2 (b) Keywords: For each vertex which represents a set of parameter values,
top layers contain the ones which represent parameter values, the bottom matching keywords can be entered in the format of tags.
layer contains the response vertices. Edges of different color constitute
the different rules.

Fig. 5: Graphical user interface of the prototype

V. I MPLEMENTATION
t =|K1 | · |K4 | + |K1 | · |K6 | + |K2 | · |K4 |
+ |K2 | · |K5 | + |K3 | · |K5 | Our prototype1 is a multi-tenant web application imple-
mented in Ruby on Rails, a web framework built on top of
=2 · 1 + 2 · 1 + 1 · 1 + 1 · 2 + 3 · 2
the object-oriented scripting language Ruby [14].
=13
The graphical user interface is backed by the Bootstrap UI
Hence, the statement modelled in the toy scenario substi- framework, and enhanced with the JavaScript diagramming
tutes 13 conventional FAQ bot questions. library mxGraph [15], [16]. Figure 5a shows the interactive
C. Impact of large number of keywords user interface which is used to design a contract statement
in the form of a hypergraph with vertices and edges. The
It is worth to note that with z we only estimate the upper
vertex labels have been implemented as stacked layers. The
bound, and t, the number of actual total questions covered,
2 top layers contain the parameter values, the layer on the
may be considerably lower. However, in real-world scenarios
bottom the response vertices. Hyperedges can be created by
the number of keywords (#k) can easily reach much higher
simply drawing edges between the vertices. Different edge
values than illustrated in the toy scenario, for example when
colors represent different hyperedges. The set of keywords
the keywords represent enumerations of cities or countries.
associated with a vertex can be easily updated as displayed
In Figure 4, we display varying statement expressivity
in Figure 5b.
scenarios. Concretely, we plot the maximum number of
questions that could be covered for 3 different configurations. Implementing the quality engineering principle of poka-
This value, z, is computed as defined in Equation 1, with yoke, the edge validation built into the user interface guaran-
respect to the number of keywords per vertex (#k). In order tees hypergraph validity by design [17]. When drawing the
to investigate the impact of one variable on z, we keep the edges, visual feedback helps the user to avoid logical errors
number of parameters |P | fixed to 2, and the number of and enforces the connection rules as defined in Section III-B.
response vertices |R| fixed to 3. In the 3 different scenarios, The different cases considered are displayed in Figure 6.
we set the number of vertices per parameter (#v) to 2, 3 and For seamless integration in any conversational agent,
4. the web application exposes an Application Programming
In general, we observe that the potential number of Interface (API). As response format we chose the open-
questions which can be covered by one statement grows standard, light-weight JavaScript Object Notation (JSON).
exponentially over the number of keywords per vertex (#k) The parameter-value pairs are included as query string in
at high rates. Even for the very low values of #v, |P | and the API url.
|R| chosen for the scenarios, z rapidly reaches the four-digit
range. Further, we see that a higher number of vertices per 1 Source code available at
parameter (#v) increases the growth rate significantly. https://github.com/axa-rev/reasoning-api-framework
[3] R. S. Wallace, The anatomy of A.L.I.C.E. Dordrecht: Springer
Netherlands, 01 2009, pp. 181–210.
Europe Asia Europe Asia [4] G. Mesnil, Y. Dauphin, K. Yao, Y. Bengio, L. Deng, D. Hakkani-Tur,
X. He, L. Heck, G. Tur, D. Yu, and G. Zweig, “Using Recurrent
Neural Networks for Slot Filling in Spoken Language Understanding,”
20 EUR 20 EUR IEEE/ACM Transactions on Audio, Speech, and Language Processing,
vol. 23, no. 3, pp. 530–539, 2015. [Online]. Available: http:
//ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6998838
(a) Correct (b) Wrong: Self-reference [5] D. Amodei, R. Anubhai, E. Battenberg, C. Case, J. Casper,
B. Catanzaro, J. Chen, M. Chrzanowski, A. Coates, G. Diamos,
E. Elsen, J. Engel, L. Fan, C. Fougner, T. Han, A. Y. Hannun,
B. Jun, P. LeGresley, L. Lin, S. Narang, A. Y. Ng, S. Ozair,
R. Prenger, J. Raiman, S. Satheesh, D. Seetapun, S. Sengupta,
Y. Wang, Z. Wang, C. Wang, B. Xiao, D. Yogatama, J. Zhan,
Europe Asia Europe Asia
and Z. Zhu, “Deep speech 2: End-to-end speech recognition in
english and mandarin,” CoRR, vol. abs/1512.02595, 2015. [Online].
Available: http://arxiv.org/abs/1512.02595
[6] G. López, L. Quesada, and L. A. Guerrero, “Alexa vs. siri vs. cortana
20 EUR 20 EUR vs. google assistant: A comparison of speech-based natural user
interfaces bt - advances in human factors and systems interaction,”
in Alexa vs. Siri vs. Cortana vs. Google Assistant: A Comparison
(c) Wrong: Connection within (d) Wrong: Rule already exists
of Speech-Based Natural User Interfaces BT - Advances in Human
same layer
Factors and Systems Interaction, I. L. Nunes, Ed. Cham: Springer
International Publishing, 2018, pp. 241–250.
[7] S. Janarthanam, Hands-On Chatbots and Conversational UI Devel-
Fig. 6: The prototype features a built-in edge validation for opment: Build Chatbots and Voice User Interfaces with Chatfuel,
error prevention. Dialogflow, Microsoft Bot Framework, Twilio, and Alexa Skills. Packt
Publishing, 2017.
[8] J. Zamora, “I’m sorry, dave, i’m afraid i can’t do that: Chatbot
perception and expectations,” in Proceedings of the 5th International
Conference on Human Agent Interaction, ser. HAI ’17. New
VI. C ONCLUSION York, NY, USA: ACM, 2017, pp. 253–260. [Online]. Available:
http://doi.acm.org/10.1145/3125739.3125766
We presented a top-down knowledge engineering approach [9] F. Koetter, M. Blohm, M. Kochanowski, J. Goetzer, D. Graziotin,
and S. Wagner, “Motivations, classification and model trial
which helps improve the capabilities of conversational of conversational agents for insurance companies,” CoRR, vol.
agents. As opposed to the static answers of conventional abs/1812.07339, 2018. [Online]. Available: http://arxiv.org/abs/1812.
FAQ bots, our approach enables smarter, dynamic responses. 07339
[10] G. Cameron, D. Cameron, G. Megaw, R. Bond, M. Mulvenna, S. O.
The user-friendly graphical user interface allows for rapid Neill, C. Armour, and M. Mctear, “Back to the Future : Lessons
contract statement creation and updating with no technical from Knowledge Engineering Methodologies for Chatbot Design and
skills required. The significant reduction of complexity cuts Development,” Proceedings of British HCI 2018, pp. 1–5, 2018.
[11] M. Bartoletti and L. Pompianu, “An Empirical Analysis of Smart
the cost to manage the contract statements, which represent Contracts: Platforms, Applications, and Design Patterns BT - Financial
the essential knowledge necessary to provide meaningful Cryptography and Data Security,” in Financial Cryptography and Data
responses. Security, M. Brenner, K. Rohloff, J. Bonneau, A. Miller, P. Y. A. Ryan,
V. Teague, A. Bracciali, M. Sala, F. Pintore, and M. Jakobsson, Eds.
A further plus of the presented architecture is that the Cham: Springer International Publishing, 2017, pp. 494–509.
statement rules are exposed through a generic API. This [12] A. Savelyev, “Contract law 2.0: ‘Smart’ contracts as the beginning
renders the system framework-agnostic and makes it possible of the end of classic contract law,” Information & Communications
Technology Law, vol. 26, no. 2, pp. 116–134, may 2017. [Online].
to integrate with any chatbot framework. Also, updates of Available: https://doi.org/10.1080/13600834.2017.1301036
the statement come into effect instantly. Finally, isolating [13] J.-Y. Huang, T.-A. Lin, and W.-P. Lee, “Using deep learning and
the knowledge and the input validation from the chatbot an external knowledge base to develop human-robot dialogues,” in
Proceedings of IEEE International Conference on Systems, Man, and
framework keeps the latter light and lean. Cybernetics, 10 2018, pp. 3709–3714.
In the future, we want to extend the keyword matching to [14] “Ruby on Rails Web Framework.” [Online]. Available: https:
more sophisticated methods: Fuzzy matching could render //rubyonrails.org
[15] “Bootstrap UI Framework.” [Online]. Available: http://getbootstrap.
the matching process more robust towards typos. Making com
use of word embeddings could match synonyms which are [16] “mxGraph Diagramming Library.” [Online]. Available: https://github.
not explicitly included in the keyword sets. Further, the data com/jgraph/mxgraph
[17] S. Shingo, Zero quality control: Source inspection and the poka-yoke
types of the keyword sets should be expanded to ranges system. CRC Press, 1986.
of numbers and dates. Also, we see potential to refine
the conversational flow by ranking the missing parameter
values by probability and plausibility. Eventually, a language-
agnostic architecture will be investigated.

R EFERENCES
[1] J. Weizenbaum, “ELIZA,” Commun. ACM, vol. 9, no. 1, 1966.
[2] K. M. Colby, S. Weber, and F. D. Hilf, “Artificial Paranoia,” Artificial
Intelligence, vol. 2, no. 1, pp. 1–25, 1971.

You might also like