You are on page 1of 12

Applied Intelligence 14, 65–76, 2001

°
c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands.

Interactive Case-Based Reasoning in Sequential Diagnosis

DAVID MCSHERRY
School of Information and Software Engineering, University of Ulster, Coleraine BT52 1SA,
Northern Ireland, United Kingdom
dmg.mcsherry@ulst.ac.uk

Abstract. Interactive trouble-shooting and customer help-desk support, both activities that involve sequential
diagnosis, represent the majority of applications of case-based reasoning (CBR). An analysis is presented of the
user-interface requirements of intelligent systems for sequential diagnosis. We argue that mixed-initiative dialogue,
explanation of reasoning, and sensitivity analysis are essential to meet the needs of experienced as well as novice
users. Other issues to be addressed by system designers include relevance and consistency in dialogue, tolerance
of missing data, and timely provision of feedback to users. Many of these issues have previously been addressed
by the developers of expert systems and the lessons learned may have important implications for CBR. We present
a prototype environment for interactive CBR in sequential diagnosis, called CBR Strategist, which is designed to
meet the identified requirements.

Keywords: case-based reasoning, diagnosis, user interface, explanation

1. Introduction have previously been addressed by the developers of


expert systems, and the lessons learned may have im-
In sequential diagnosis, tests are selected at each stage portant implications for interactive CBR.
of the evidence-gathering process on the basis of their For example, an expert system for sequential diag-
usefulness for discriminating between competing hy- nosis can explain the relevance of a selected test at any
potheses that may account for a reported symptom or stage of the evidence-gathering process [6]. In inter-
fault. The usefulness of a test, for example as measured active CBR, the details of a target problem are sim-
by the expected reduction in the entropy of the distri- ilarly elicited from the user, often with questions se-
bution of disease or solution probabilities, depends on lected or ranked in order of usefulness by the system
the evidence, if any, previously reported. Methods used [4, 5, 7], However, the relevance of a test selected by
to guide test selection in intelligent systems include nearest-neighbour retrieval, or inductive retrieval based
information-theoretic measures [1], rule-based reason- on information-theoretic measures, can be difficult to
ing [2], and value of information [3]. Often the aim in explain. An algorithm for decision-tree induction in
sequential diagnosis is to minimize the number of tests which attribute selection can be explained in strategic
required to reach a firm diagnosis and hence reduce the terms [8] is the basis of an approach to interactive CBR
risks and costs associated with testing. Another impor- proposed in this paper. The algorithm, called Strate-
tant issue to be addressed by system designers is the gist, is adapted from a model of the evidence-gathering
system’s acceptability to users. strategies used by doctors [9–13].
Interactive trouble-shooting and customer help-desk A demand-driven (or lazy) version of Strategist has
support, both activities that involve sequential diagno- been implemented in an environment providing inte-
sis, represent the majority of applications of case-based grated support for machine learning, problem solving
reasoning (CBR) [4, 5]. Many of the issues related to and explanation [8]. This paper presents an extended
problem-solving efficiency and acceptability to users version of the environment, called CBR Strategist,
66 McSherry

which is designed as a tool for interactive CBR in se- own experience, knows which features of the problem
quential diagnosis. Features of the extended environ- are most relevant.
ment include mixed-initiative dialogue, visual feed- Another aspect of mixed-initiative dialogue is
back on the impact of evidence reported, sensitivity whether the user can volunteer an opinion. A system
analysis, and an approach to the maintenance of con- which ignores the user’s opinion is missing an oppor-
sistency in dialogue in which there is no requirement tunity to involve the user more closely in the problem-
for explicit domain knowledge. solving process and may also be missing an opportunity
An analysis of the user-interface requirements of in- to solve the problem more efficiently. Often in fault di-
telligent systems for sequential diagnosis, from both agnosis, for example, a professional user may have a
expert systems and CBR perspectives, is presented in good idea about what is causing the problem and is us-
Section 2. The algorithm that provides the basis for in- ing the system as a means of confirming her opinion.
ductive retrieval and explanation of reasoning in CBR Clinical decision-support systems which recognize that
Strategist is described in Section 3. The decision tree the user may wish to suggest a diagnosis to be consid-
it induces is often identical to the tree induced by ID3 ered include DXplain [15] and Hypothesist [11].
[14] when the number of attributes is small [8]. As
shown in this paper, it is also possible for the decision
trees to differ significantly in structure. An overview 2.2. Feedback on the Impact of Reported Evidence
of CBR Strategist is presented in Section 4, and an ex-
ample case library is used to illustrate its use as a tool Intelligent systems for sequential diagnosis should pro-
for interactive CBR in the domain of computer fault vide users with timely feedback on the impact of re-
diagnosis. ported evidence. As each piece of evidence is reported,
some diagnoses become more likely and others less
likely. In problem-solving dialogue with a human ex-
2. The User Interface in Sequential Diagnosis
pert, it would be unusual not to receive some feedback
as evidence is reported, such as a comment that a re-
Aspects of the user interface in intelligent systems for
ported symptom rules out a particular diagnosis. Simi-
sequential diagnosis which may affect their problem-
larly, the user of an intelligent system should not have
solving efficiency and acceptability to users include
to wait to the end of a consultation, when only a single
mixed-initiative dialogue, feedback on the impact of
possibility remains, to obtain feedback.
reported evidence, dialogue relevance and consistency,
Continuous updating and display of similarity scores
explanation of reasoning, tolerance of missing data, and
(or rankings) of retrieved cases as questions are an-
sensitivity analysis.
swered by the user is one way to provide incremental
feedback in interactive CBR [5, 7]. This approach is
2.1. Mixed-Initiative Dialogue similar to the incremental updating and display of dis-
ease probabilities in clinical decision support [1, 9].
The importance of mixed-initiative dialogue is well rec- Hypothesist [11] provides visual feedback in the form
ognized by the developers of expert systems and clini- of a graphical display, continually updated in the light
cal decision-support systems [9, 11, 15–17]. Intelligent of new evidence, of disease probabilities. An exam-
systems are unlikely to be accepted by users, and pro- ple of visual feedback in interactive CBR is the colour
fessional users in particular, if they insist on asking the coding of user preferences in a knowledge-navigation
questions, and force the user to play a passive role. In- system for exploring the domain of new cars [18]. Users
stead, the user should be able to volunteer data at any can adjust their preferences interactively and the result-
stage of the consultation dialogue. ing changes in the degree of match of displayed alter-
Some tools for interactive CBR support a form of natives are immediately reflected in their associated
mixed-initiative dialogue in which the user can select colour patterns.
from a list of questions to be answered and not just the
question considered most useful by the system [4, 5, 7].
Enabling the user to volunteer data may be important 2.3. Relevant Dialogue
not only because it is more acceptable to users but also
to increase efficiency in problem solving. It may be that The questions an intelligent system asks the user should
the user, reminded of a similar previous case from her be relevant in the context of the problem or fault
Interactive Case-Based Reasoning in Sequential Diagnosis 67

reported by the user. Inevitably, some questions will their conditions are satisfied [22–24]. A similar rule-
be relevant for certain problems and not for others. based approach has been used in CBR [4, 5, 7]. In
In computer fault diagnosis, if the problem reported conversational CBR, the problem of inferring aspects
is that there are strange characters on the screen, then of a user’s problem from its partial description, thereby
asking the user if the power light is on is clearly an ir- avoiding potentially inconsistent questions and increas-
relevant question. One reason for the popularity of the ing retrieval efficiency, is referred to as dialogue infer-
backward-chaining (or goal-driven) inference mecha- encing. Recently, a model-based approach to dialogue
nism widely used in expert systems is that the user is inferencing has been proposed as a means of avoid-
asked only questions that are relevant in the context of ing the overheads associated with the acquisition and
the system’s current goal. There is no reason to suppose maintenance of rules [4, 7].
that relevant dialogue is any less important to users in In Section 4, an approach to the maintenance of dia-
interactive CBR. One approach to ensuring relevant di- logue consistency in interactive CBR based on induc-
alogue in CBR relies on a heterogeneous case structure tive retrieval is presented in which there is no require-
in which only attributes (or questions) which are rele- ment for explicit domain knowledge.
vant in the diagnosis of the fault explained by a case
are associated with the case [5, 7].
This is the approach used in NaCoDAE [4, 7], a tool 2.5. Explanation of Reasoning
for conversational CBR [19]. In conversational CBR,
the user enters a partial description of her problem in There is general agreement that to be useful, intelligent
natural language and the system assists in further elab- systems must be capable of explaining their reasoning
orating the problem through a conversation with the [6, 25, 26]. An advantage of CBR systems in compari-
user. In NaCoDAE, the user’s description of the prob- son with expert systems is their ability to justify a solu-
lem is used to retrieve cases with similar descriptions tion in terms of actual experience, which may be more
provided by the authors of the case library. To provide acceptable to users than an explanation based on rules.
further details of the problem, the user selects from a list However, justification of a solution on the grounds that
of unanswered questions in the most similar retrieved it worked before loses some of its appeal when there is
cases, ranked in order of their frequency. no exact match for a target case. Solution justification
Maximizing information gain is another method must therefore take account of differences between the
used to rank questions in conversational CBR [20, 21]. target case and the retrieved case as well as similarities
[27].
Justification of a CBR solution is analogous to the
2.4. Consistent Dialogue explanation provided by an expert system, usually con-
sisting of a reasoning trace, when asked at the end of a
A problem related to dialogue relevance is how to avoid consultation how a conclusion was reached. In sequen-
asking the user a question when the answer can be in- tial diagnosis, an expert system can also explain the
ferred from the user’s initial description of the prob- relevance of a selected test at any stage of the evidence-
lem or answers to previous questions [4, 7]. In printer gathering process. The explanation provided by the ex-
trouble-shooting, for example, there is no point in ask- pert system is usually a simple statement of its current
ing if the user is having print quality problems if she has goal and the rule it is trying to fire. Less commonly, an
already complained of black spots on the page. Asking expert system may be able to explain its reasoning in
such unnecessary questions not only reduces problem- strategic terms [6].
solving efficiency; it also reveals a lack of common Explanation of question relevance is equally desir-
sense that is likely to reduce the user’s confidence in able in CBR, often in which the details of a target
the system [22]. Worse still, it leaves open the possi- problem are elicited interactively from the user and
bility of an inconsistent response, perhaps leading to questions are selected at the system’s initiative. In
the system being unable to reach a solution or even decision-tree induction, a rule-like explanation of an
suggesting an incorrect solution. attribute’s relevance can be generated by looking ahead
Inconsistent dialogue can be avoided in an ex- to find the shortest path from the current node of the
pert system with a mixed inference strategy in which decision tree to a leaf node [28]. A similar technique
backward-chaining is combined with opportunistic for- could be used to explain the relevance of questions
ward chaining based on priority rules that fire as soon as in interactive CBR based on inductive retrieval. An
68 McSherry

alternative approach described in this paper is based ing the next best attribute at the current node when an
on an algorithm for decision-tree induction in which attribute’s value is unknown at problem-solving time.
attribute relevance can be explained in strategic terms
[8]. The strategic explanations thus provided during
the evidence-gathering process are intended to comple- 2.7. Sensitivity Analysis
ment, rather than replace, the precedent-based justifi-
cation of solutions already provided, at least implicitly, Although sensitivity analysis is easily supported in
by CBR systems. an intelligent system that already supports mixed-
initiative dialogue, its importance is often understated
[12, 35]. Because one source of uncertainty in diagno-
2.6. Tolerance of Missing Data sis is uncertain data, it is important that the user should
be able to examine the effects of changing the value
Intelligent systems for sequential diagnosis must be of an attribute about which she is uncertain, or the an-
able to reason effectively in the absence of complete swer to a question that involves subjective judgement.
data. Often there are questions that the user is unable to Another source of uncertainty is incomplete data. For
answer, for example because an observation was not example, the user may wish to examine the potential
recorded and cannot be repeated, or because the answer effects of tests whose results are unknown, or which
requires an expensive or complicated test that the user carry high risk or cost. A simple what-if analysis may
is reluctant or incompetent to perform. In interactive be enough to show that the missing data can have lit-
CBR, allowing the user to select from a list of ques- tle effect on the outcome, thus increasing the user’s
tions rather than answering the one considered most confidence in the solution.
useful by the system is one way to ensure that progress
can be made when the user is unable to answer every
3. Decision-Tree Induction in Strategist
question [4, 5, 7]. A retrieval strategy that focuses ini-
tially on features with zero cost is another approach
Strategist is an algorithm for top-down induction of
which recognizes that certain questions may be easier
decision trees in which attribute relevance can be ex-
(or less costly) to answer [29].
plained in strategic terms [8]. Its attribute-selection
In systems that take the initiative by asking direct
strategies, such as confirming a target outcome class
questions, the user should have the option to answer
or eliminating a competing outcome class, are based
unknown to any question. This is a common feature in
on the evidence-gathering strategies used by doctors in
an expert system, which can continue in the absence
diagnosis [36, 37]. Continually revised as the data set
of the requested information by trying another rule or
is recursively partitioned, the target outcome class is
attempting to prove a different goal.
initially the one that is most likely in the data set.
In CBR, nearest-neighbour retrieval is less sensi-
tive to missing data than inductive retrieval [5]. Auriol
et al. [30] propose the integration of inductive and 3.1. Attribute-Selection Strategies
nearest-neighbour retrieval as an approach to tolerat-
ing missing data in a decision tree. Approaches that rely In order of priority, Strategist’s main attribute-selection
purely on decision-tree technology include those im- strategies are:
plemented in CART [31] and C4.5 [32]. Quinlan [33]
compared several approaches to tolerating missing val- CONFIRM confirm the target outcome class
ues in decision-tree induction and concluded that while ELIMINATE eliminate the likeliest alternative
some were clearly inferior, none was uniformly supe- outcome class
rior to the rest. VALIDATE increase the probability of the target
Any algorithm for top-down induction of decision outcome class
trees can be modified to anticipate the problem of miss- OPPOSE reduce the probability of the likeliest
ing data by building a larger tree in which each node has alternative outcome class
an additional branch to be followed when the value of
the most useful attribute is unknown [28]. In algorithms A fifth strategy called DISCRIMINATE is available
that adopt a demand-driven approach to decision-tree but only occasionally needed in practice. In the cur-
induction [8, 28, 29, 34], this is equivalent to select- rent subset of the data set, an attribute will support the
Interactive Case-Based Reasoning in Sequential Diagnosis 69

CONFIRM strategy if it has a value which occurs only where el(E i , ξ ) is the eliminating power of E i in S
in the target outcome class, the ELIMINATE strategy and ξ is the combination of attribute values on the path
if one of its values occurs in the target outcome class from the root node to S.
but not in the likeliest alternative outcome class, the When the available strategy of highest priority is
VALIDATE strategy if one of its values is more likely VALIDATE or OPPOSE, the attribute selected is the
in the target outcome class than in any other outcome one for which the expected weight of evidence in favour
class, and the OPPOSE strategy if one of its values is of the target outcome class, relative to the likeliest al-
less likely in the likeliest alternative outcome class than ternative, is greatest. In a given subset S of the data set,
in any other outcome class. the expected weight of evidence of an attribute A in
When more than one attribute is available to support favour of an outcome class C1 , relative to an alterna-
the CONFIRM or ELIMINATE strategy, the attribute tive outcome class C2 , is:
selected is the one whose expected eliminating power
in favour of the target outcome class is greatest. The Xn
p(E i | C1 , ξ ) p(E i | C1 , ξ )
ψ(A, C1 , C2 , ξ ) =
eliminating power of an attribute value E is the sum i=1
p(E i | C2 , ξ )
of the probabilities of the outcome classes surviving in
the current subset which are eliminated by the attribute where E 1 , E 2 , . . . , E n are the values of A and ξ is the
value, or zero if none is eliminated. In a given subset combination of attribute values on the path from the
S of the data set, the expected eliminating power of an root node to S.
attribute A with values E 1 , E 2 , . . . , E n , in favour of an
outcome class C, is:
3.2. Example Data Set
X
n
γ (A, C, ξ ) = p(E i | C, ξ ) el(E i , ξ ) Table 1 shows an artificial data set based on a highly
i=1 simplified version of the real-world problem of fault

Table 1. An artificial data set for diagnosis of the computer fault “screen is dark.”

Power Fan Computer Computer Monitor Brightness Monitor


light can be switched plugged light level switched Possible
on heard on in on adjusted on solution

false false true true false false false FPC


false false true true false false true FPC
false false true true false true false FPC
false false true true false true true FPC
false false false true false false false CSO
false false false true false false true CSO
false false false true false true false CSO
false false false true false true true CSO
false false true false false false false CNPI
false false true false false false true CNPI
false false true false false true false CNPI
false false true false false true true CNPI
true true true true false false false MSO
true true true true false true false MSO
true true true true false false true MNPI
true true true true false true true MNPI
true true true true true false true BLTL
true true true true true true true VCD
70 McSherry

diagnosis in a computer system. The fault reported by Similarly, the expected eliminating power of fan-
the user is assumed to be that the screen is dark. Out- can-be-heard is 0.60. Since the two attributes are
come classes in the data set are faulty power cord (FPC), equally good supporters of the CONFIRM strategy,
computer switched off (CSO), computer not plugged power-light-on is arbitrarily selected as the attribute
in (CNPI), monitor switched off (MSO), monitor not to partition the current subset. In the partition cor-
plugged in (MNPI), brightness level too low (BLTL), responding to power-light-on = true, the target out-
and video cable disconnected (VCD). Of course, there come class changes to MSO. Among the examples
are other possible reasons why the screen may be dark, in this subset, monitor-switched-on has a value which
such as screen-saver running, monitor in power-saving occurs only in the target outcome class. As the only
mode, a blown fuse, or an interruption to the power attribute available to support the CONFIRM strat-
supply. egy, it is used to partition the current subset. Sim-
ilarly, a single attribute is available to support the
CONFIRM strategy at each of the remaining non-leaf
3.3. Induced Decision Trees
nodes.
Only two of the available attribute-selection stra-
Decision-tree induction in Strategist is illustrated in
tegies, CONFIRM and ELIMINATE, are required to
Fig. 1. The likeliest outcome class in each subset (or an
induce the entire tree and a quantitative measure
arbitrarily selected outcome class where two or more
of attribute usefulness is needed at only one of the
are equally likely) is shown in bold. The likeliest alter-
nodes.
native outcome class is also shown. Initially the likeli-
The decision tree induced by ID3 from the example
est outcome class in the example data set is FPC and
data set is shown in Fig. 2. It differs from the Strate-
the likeliest alternative is CSO. None of the available
gist tree both in structure and in the attribute selected
attributes has a value which occurs only in the target
to appear at the root node. According to the infor-
outcome class, so the CONFIRM strategy is not sup-
mation gain criterion [14], the most useful attribute
ported in the data set. However, the attribute computer-
is power-light-on. In Strategist, power-light-on would
switched-on has a value which occurs in FPC but not in
support the VALIDATE strategy, since power-light-on
CSO and will therefore support the ELIMINATE strat-
= false is more likely (or at least equally likely) in
egy. As the only such attribute, computer-switched-on
FPC than in any other outcome class and, if known,
is selected to appear at the root node of the decision tree.
increases its probability from 0.22 to 0.33. However,
As CSO is the only outcome class surviving in the
as it always has the same value in FPC and CSO,
subset of the data set for which computer-switched-on
power-light-on can support neither of the CONFIRM
= false, the corresponding node of the decision tree
and ELIMINATE strategies which are given priority in
is a leaf node. In the subset with computer-switched-
Strategist.
on = true, the target outcome class remains unchanged
When the trees induced by the two algorithms dif-
but the likeliest alternative changes to CNPI. Again no
fer, as in this example, the ID3 tree tends to be smaller
attribute is available to support the CONFIRM strat-
[8]. In the Strategist tree, the average number of ques-
egy, but the attribute computer-plugged-in has a value
tions required to reach a leaf node is approximately 4
that will eliminate CNPI. Following its elimination,
compared with an average of 3 for the ID3 tree. How-
the likeliest alternative changes to MSO. However,
ever, the advantage of the Strategist tree is that the rel-
power-light-on and fan-can-be-heard now have values
evance of the attribute selected at each node can be
that occur only in FPC and will therefore support the
explained in strategic terms. For example, when power-
CONFIRM strategy.
light-on is selected by Strategist it is because the algo-
In the current subset of the data set, the expected eli-
rithm is attempting to confirm FPC, the target outcome
minating power of power-light-on in favour of FPC is:
class.
γ (power-light-on, FPC, ξ )
= p(power-light-on = true | FPC, ξ ) × p(FPC | ξ ) 4. CBR Strategist
+ p(power-light-on = false | FPC, ξ )
Written in Prolog for the Apple Macintosh, CBR Strate-
× ( p(MSO | ξ ) + p(MNPI | ξ ) + p(BLTL | ξ ) gist is a prototype environment for interactive CBR
+ p(VCD | ξ )) = 0.60 in sequential diagnosis in which the user interface is
Interactive Case-Based Reasoning in Sequential Diagnosis 71

Figure 1. Decision tree induced by Strategist from the example data set.

designed to meet the requirements identified in come class or diagnosis. Case retrieval is based on the
Section 2. A case library based on the example data attribute-selection strategies described in Section 3.
set from Section 3 is used here to illustrate the user-
interface features provided in the environment and its 4.1. Mixed-Initiative Dialogue
use as a tool for interactive CBR in the domain of
computer fault diagnosis. The case structure in CBR At the start of a consultation, the user can volunteer
Strategist is homogeneous, with each case represented as much information as she wishes by selecting from
as a list of attribute-value pairs and an associated out- a question menu (Fig. 3), and answering the selected
72 McSherry

Figure 2. Decision tree induced by ID3 from the example data set.

remains in force, however unlikely, until revised by the


user or eliminated by the reported evidence.
The user can recapture the initiative, or change the
target outcome class, at any stage of the consultation.

4.2. Visual Feedback

CBR Strategist provides visual feedback in the form of


a graphical display, continually updated in the light of
new evidence, of the relative frequencies of surviving
cases. Fig. 4 shows the relative frequencies of surviving
cases in the example case library when the computer is
Figure 3. In CBR Strategist, the user can volunteer data at any known to be switched on and plugged in.
stage.
4.3. Relevant Dialogue
question. There is no ranking of questions in order of
usefulness in the question menu, to which the user When given the initiative, CBR Strategist’s selection
can return at any stage to volunteer additional data of questions based on the CONFIRM, ELIMINATE,
or correct an answer to a previous question. The an- VALIDATE and OPPOSE strategies described in
swers to previous questions are displayed in a sep- Section 3 ensures the relevance of questions the user
arate window. As each question is answered, CBR is asked. Its strategic approach to inductive retrieval
Strategist uses the evidence provided to eliminate cases is based on a demand-driven version of Strategist [8].
whose values for the attribute differ from the reported Instead of partitioning the current subset of the case
value. library into subsets corresponding to all values of a se-
The user can continue selecting questions she wishes lected attribute, it asks the user for the attribute’s value
to answer until only a single outcome class remains, and uses the evidence provided to eliminate cases in
or pass the initiative to CBR Strategist at any stage the current subset whose values for the attribute differ
by clicking on the “CBR” button. Before giving CBR from the reported value.
Strategist the initiative, she can volunteer an opinion
about the cause of the problem by clicking on the 4.4. Maintenance of Dialogue Consistency
“Goal” button and choosing a target outcome class.
Alternatively, she can leave the choice of target out- Inconsistent dialogue can occur only if inconsistent
come class to CBR Strategist. If so, the target outcome data is reported by the user and accepted by the system.
class (initially the likeliest outcome class given any ev- To maintain consistency in dialogue, it is therefore suf-
idence already reported) is continually revised in the ficient to ensure that inconsistent data, if reported by
light of new evidence and the user is informed when the user, is immediately rejected by the system. This
it changes. A target outcome class selected by the user is the approach used in CBR Strategist. Interestingly,
Interactive Case-Based Reasoning in Sequential Diagnosis 73

Figure 4. In CBR Strategist, the relative frequencies of surviving cases are continually updated as new evidence is reported.

there is no requirement for explicit domain knowledge question to which one of the possible answers is incon-
in the approach. sistent with evidence previously reported. However, if
A potential source of inconsistency arises, for ex- the inconsistent answer is selected by the user, CBR
ample, when the value of one case attribute logically Strategist will once again inform the user that there are
depends on the value of another. One such logical de- no matching cases.
pendency in the example case library is the common-
sense rule: 4.5. Explanation of Reasoning
if power-light-on = true
As in an expert system, the user can query the relevance
then computer-switched-on = true of any question the system asks. CBR Strategist’s ex-
planation, like that of its machine-learning predecessor
Though not explicitly known by CBR Strategist, the
[8] depends on its current strategy. For example, in the
rule is implicit in the case data and this is enough
CONFIRM strategy, the user is shown the value of the
for the potential inconsistency to be avoided. For ex-
selected attribute that will confirm the target outcome
ample, suppose that the user has previously reported
class. In the ELIMINATE strategy, the user is shown
that the power light is on. Following the elimination
the value of the selected attribute that will eliminate the
of cases with power-light-on = false, there can be no
likeliest alternative outcome class.
case with computer-switched-on = false among the sur-
viving cases in the case library. It follows that if the
user now reports that the computer is not switched on, 4.6. Tolerance of Missing Data
CBR Strategist will be unable to find a matching case
among the surviving cases. It therefore informs the user The user can answer unknown to any question, in which
that there are no matching cases, and suggests that she case CBR Strategist simply selects the next most useful
should check the consistency of the reported data. For question; that is, the one that best supports the attribute-
example, if the user has mistakenly reported that the selection strategy of highest priority now available. Of
power light is on, she can recover from the error by us- course, a unique classification or diagnosis may not be
ing the question menu to correct her previous answer. possible if the user is unable to provide sufficient data.
As this example illustrates, inconsistent data can be In that case, the user may decide to adopt the likeli-
seen as an attempt to describe a point in the problem est solution as suggested by the relative frequencies of
space that cannot exist in reality. Thus provided there partially matching cases.
is no inconsistency in the case library, the simple ap-
proach described ensures that inconsistent data is never 4.7. Sensitivity Analysis
accepted; for if any reported finding is inconsistent with
evidence previously reported, there can be no matching At any stage of a consultation, the user can return to
case in the case library. If some of the case attributes are the question menu (Fig. 3) to examine the effects of
non-binary, it is possible that CBR Strategist may ask a changing an uncertain answer, or the possible effects of
74 McSherry

has been confirmed. Because of the limited coverage of


the example case library, CBR Strategist is unaware of
other possible causes of the reported symptoms, such
as a blown fuse or an interruption to the power supply.

4.9. Discussion

Features of the user interface in CBR Strategist include


mixed-initiative dialogue, visual feedback on the im-
pact of reported evidence, tolerance of missing data,
Figure 5. In CBR Strategist, the user can answer unknown to any
question. sensitivity analysis, and maintenance of dialogue con-
sistency. Its strategic approach to inductive retrieval
ensures the relevance of dialogue and enables the rea-
an omitted test that carries high risk or cost. The relative
soning process to be explained in strategic terms. Al-
frequencies of matching cases (Fig. 4) are dynamically
though no account is taken of the relative costs of tests
updated to enable the user to visualize the effects of
in its attribute-selection strategies, they could easily be
changes in the reported evidence.
modified to give priority to least expensive tests.
CBR Strategist’s approach to the maintenance of di-
4.8. Example Consultation alogue consistency ensures that inconsistent data, if
reported by the user, is immediately rejected. A limi-
Before giving CBR Strategist the initiative, the user has tation of the approach is that the system is unable to
reported that the computer is switched on and plugged determine whether the absence of a matching case is
in. On receiving the initiative, CBR Strategist informs due to lack of problem-space coverage or inconsistency
the user that the target outcome class is FPC, and its in the evidence reported by the user. However, the over-
first question, selected for its ability to support the heads associated with the acquisition or discovery of
CONFIRM strategy, is whether the power light is on the knowledge required for this distinction to be made
(Fig. 5). The user is unable to answer this question, are unlikely to be justified in practice. If the user is
so CBR Strategist looks for the next best question. satisfied that the reported data is consistent, then the
As noted in Section 3, power-light-on and fan-can-be- absence of a matching case must mean that the target
heard are equally good supporters of the CONFIRM case is not covered by the case library. Pending the ad-
strategy when the computer is switched on and plugged dition of a new case to the case library, adopting the
in and FPC is the target outcome class. likeliest solution as suggested by the relative frequen-
When asked if the fan can be heard, the user queries cies of partially matching cases remains an immediate
the relevance of the question (Fig. 6). The explanation option for the consultation user.
generated by CBR Strategist is shown in Fig. 7. CBR The most serious limitation of CBR Strategist, as
Strategist now repeats the question. The user replies currently implemented, arises from its assumption of
that the fan cannot be heard, and is informed that FPC a homogeneous case structure [5] in which the value
of every attribute is recorded for every case. This es-
sentially limits its scope to the diagnosis of a single
reported fault, since the attributes which are relevant in
the diagnosis of other faults are unlikely to be the same.
A similar limitation is shared by many algorithms for
machine learning which assume a homogeneous struc-
ture in the data sets to which they are applied. However,
to be useful for interactive trouble-shooting, by far the
most extensive use of CBR technology [4], a CBR tool
needs to cover an acceptable range of reported faults.
An approach to providing coverage for a range of
Figure 6. Querying the relevance of a question selected by CBR faults in CBR Strategist is being investigated in which
Strategist. a homogeneous case structure is assumed only within
Interactive Case-Based Reasoning in Sequential Diagnosis 75

Figure 7. Explanation of reasoning in CBR Strategist.

partitions of the case library corresponding to differ- logical dependencies implicit in the case data are suf-
ent faults. While cases in different partitions may be ficient to ensure the immediate rejection of a reported
indexed by different attributes, every case will also be finding that is inconsistent with evidence previously
indexed by the reported fault (e.g., screen is dark) ex- reported. An approach to providing trouble-shooting
plained by its solution (e.g., faulty power cord). At the coverage for a range of faults in CBR Strategist is be-
start of a consultation, the user will select from the ing investigated in which a homogeneous case structure
faults covered by the case library, triggering the re- is assumed only within partitions of the case library
trieval of all cases in the corresponding partition of the corresponding to different faults.
case library. From that point, the consultation will con-
tinue as in the current version of CBR Strategist. Sim- Acknowledgments
ilar functionality is provided in HOMER, a help-desk
application in which the user can start a consultation The author is grateful to the reviewers for their insight-
by selecting from a problem hierarchy [38]. ful comments and suggestions.
Automating the refinement of case libraries to in-
crease their conformity with design guidelines is a topic References
of increasing research interest [19]. While the restruc-
turing of an existing case library to conform with the 1. G.A. Gorry, J.P. Kassirer, A. Essig, and W.B. Schwartz, “Deci-
structure proposed here should not be difficult to auto- sion analysis as the basis for computer-aided management of
mate, a more challenging problem will be how to deal acute renal failure,” American Journal of Medicine, vol. 55,
pp. 473– 484, 1973.
with the residual heterogeneity that is likely to remain
2. E.H. Shortliffe, “Clinical decision support systems,” in Medical
within resulting partitions of the case library. Modi- Informatics: Computer Applications in Health Care, edited by
fication of CBR Strategist’s attribute-selection strate- E.H. Shortliffe and L.E. Perreault, Addison-Wesley: Reading,
gies to tolerate missing values in library cases is one Massachusetts, pp. 466–502, 1990.
of the possible solutions to be investigated by further 3. D.E. Heckerman, Probabilistic Similarity Networks, MIT Press:
Cambridge, Massachusetts, 1991.
research.
4. D.W. Aha, “The omnipresence of case-based reasoning in sci-
ence and application,” Expert Update, vol. 1, pp. 29–45, 1998.
5. Summary 5. I. Watson, Applying Case-Based Reasoning: Techniques for En-
terprise Systems, Morgan Kaufmann: San Francisco, California,
1997.
Following an analysis of the user-interface require-
6. R.W. Southwick, “Explaining reasoning: an overview of expla-
ments of intelligent systems for sequential diagnosis, a nation in knowledge-based systems,” Knowledge Engineering
prototype environment for interactive CBR in sequen- Review, vol. 6, pp. 1–19, 1991.
tial diagnosis has been presented. Features of the user 7. D.W. Aha, T. Maney, and L.A. Breslow, “Supporting dialogue
interface in CBR Strategist include mixed-initiative di- inferencing in conversational case-based reasoning,” in Proceed-
ings of the Fourth European Workshop on Case-Based Reason-
alogue, visual feedback on the impact of reported evi-
ing, Dublin, Ireland, 1998, pp. 262–273.
dence, tolerance of missing data, and sensitivity analy- 8. D. McSherry, “Strategic induction of decision trees,” in Pro-
sis. Inductive retrieval based on the evidence-gathering ceedings of ES98, Cambridge, England, 1998, pp. 15–26. Also
strategies used by doctors enables CBR Strategist to to appear in Knowledge-Based Systems.
explain the relevance of questions it asks the user in 9. D. McSherry, “Intelligent dialogue based on statistical models of
clinical decision making,” Statistics in Medicine, vol. 5, pp. 497–
strategic terms.
502, 1986.
A simple approach to the maintenance of consis- 10. D. McSherry, “A domain-independent theory for testing fault
tency in dialogue has been presented in which there is hypotheses,” in Colloquium on Intelligent Fault Diagnosis, In-
no requirement for explicit domain knowledge. Instead, stitution of Electrical Engineers: Londons, 1992.
76 McSherry

11. D. McSherry, “HYPOTHESIST: A development environment 28. D. McSherry, “Integrating machine learning, problem solving
for intelligent diagnostic systems,” in Proceedings of the Sixth and explanation,” in Proceedings of ES95, Cambridge, England,
Conference on Artificial Intelligence in Medicine Europe, 1995, pp. 145–157.
Grenoble, France, 1997, pp. 223–234. 29. B. Smyth and P. Cunningham, “A comparison of incremental
12. D. McSherry, “Avoiding premature closure in sequential diag- case-based reasoning and inductive learning,” in Proceedings
nosis,” Artificial Intelligence in Medicine, vol. 10, pp. 269–283, of the Second European Workshop on Case-Based Reasoning,
1997. Chantilly, France, 1994, pp. 151–164.
13. D. McSherry, “Dynamic and static approaches to clinical data 30. E. Auriol, M. Manago, K.-D. Althoff, S. Wess, and S. Dittrich,
mining,” Artificial Intelligence in Medicine, vol. 16, pp. 97–115, “Integrating induction and case-based reasoning: Methodolog-
1999. ical approach and first evaluation,” in Proceedings of the Sec-
14. J.R. Quinlan, “Induction of decision trees,” Machine Learning, ond European Workshop on Case-Based Reasoning, Chantilly,
vol. 1, pp. 81–106, 1986. France, 1994, pp. 18–32.
15. G.O. Barnett, J.J. Cimino, J.A. Hupp, and E.P. Hoffer, “DXplain: 31. L. Breiman, J.H. Friedman, and C.J. Stone, Classification and
an evolving diagnostic decision-support system,” Journal of the Regression Trees, Wadsworth: Pacific Grove, California, 1984.
American Medical Association, vol. 258, pp. 67–74, 1987. 32. J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan
16. D.C. Berry and D.E. Broadbent, “Expert systems and the man- Kaufmann: San Mateo, California, 1993.
machine interface. Part Two: The user interface,” Expert Sys- 33. J.R. Quinlan, “Unknown attribute values in induction,” in Pro-
tems, vol. 4, pp. 18–28, 1987. ceedings of the Sixth International Workshop on Machine Learn-
17. R.S. Patil, P. Szolovits, and W.B. Schwartz, “Modeling knowl- ing, Ithaca, New York, 1989, pp. 164–168.
edge of the patient in acid-base and electrolyte disorders,” in 34. J.H. Friedman, R. Kohavi, and Y. Yun, “Lazy decision trees,” in
Artificial Intelligence in Medicine, edited by P. Szolovits, West- Proceedings of the Thirteenth National Conference on Artificial
view Press, Boulder, Colorado pp. 191–226, 1982. Intelligence, Portland, Oregon, 1996, pp. 717–724.
18. K.J. Hammond, R. Burke, and K. Schmitt, “A case-based ap- 35. R.M. O’Keefe, V. Belton, and T. Ball, “Experiences using ex-
proach to knowledge navigation,” in Case-Based Reasoning: Ex- pert systems in O.R.” Journal of Operational Research Society,
periences, Lessons & Future Directions, edited by D.B. Leake, vol. 37, pp. 657–668, 1986.
AAAI Press/MIT Press, Menlo Park, California, pp. 125–136, 36. A.S. Elstein, L.A. Schulman, and S.A. Sprafka, Medical Prob-
1996. lem Solving: An Analysis of Clinical Reasoning, Harvard Uni-
19. D.W. Aha and L.A. Breslow, “Refining conversational case li- versity Press, Cambridge, Massachusetts, 1978.
braries,” in Proceedings of the Second International Conference 37. E.H. Shortliffe and G.O. Barnett, “Medical data: their acqui-
on Case-Based Reasoning, Providence, Rhode Island, 1997, sition, storage and use,” in Medical Informatics: Computer
pp. 267–278. Applications in Health Care, edited by E.H. Shortliffe and
20. H. Shimazu, A. Shibata, and K. Nihei, (2000) “ExpertGuide: a L.E. Perreault, Addison-Wesley: Reading Massachusetts,
conversational case-based reasoning tool for developing men- pp. 37–69, 1990.
tors in knowledge spaces,” Applied Intelligence, vol. 14, no. 1, 38. M. Göker, T. Roth-Berghofer, R. Bergmann, T. Pantleon,
pp. 33–48, 2001. R. Traphöner, S. Wess, and W. Wilke, “The development of
21. Q. Yang and J. Wu, (2000) “Enhancing the effectiveness of HOMER: A case-based CAD/CAM help-desk support tool,” in
interactive case-based reasoning with clustering and decision Proceedings of the Fourth European Workshop on Case-Based
forests,” Applied Intelligence, vol.14, no.1, pp. 49–64, 2001. Reasoning, Dublin, Ireland, 1998, pp. 346–357.
22. P. Jackson, Introduction to Expert Systems, Addison-Wesley, (Ist
Edition) Wokingham, UK, 1986.
23. J. Cendrowska and M. Bramer, “Inside an expert system: A ratio-
nal reconstruction of the MYCIN consultation system,” in Artifi-
cial Intelligence: Tools, Techniques and Applications, edited by
T. O’Shea and M. Eisenstadt, Harper & Row, NewYork, pp. 453–
497, 1984.
24. D. McSherry and K. Fullerton, “PRECEPTOR: a shell for med-
ical expert systems and its application in a study of prognostic
indices in stroke,” Expert Systems, vol. 2, pp. 140–147, 1985.
25. A.K. Goel and J.W. Murdock, “Meta-cases: explaining case-
based reasoning,” in Proceedings of the Third European Work- David McSherry graduated with first class honours in Mathematics
shop on Case-Based Reasoning, Lausanne, Switzerland, 1996, at the Queen’s University of Belfast in 1973, and was awarded the
pp. 150–163. degrees of M. Sc. and Ph. D. by the same university in 1974 and 1976.
26. D.B. Leake, “CBR in context: The present and future,” in Case- Currently a senior lecturer in Computing Science at the University of
Based Reasoning: Experiences, Lessons & Future Directions, Ulster, his research interests include case-based reasoning, machine
edited by D.B. Leake, AAAI Press/MIT Press, Menlo Park, Cal- learning, and artificial intelligence in medicine. He has published a
ifornia, pp. 3–30, 1996. large number of papers in the proceedings of international confer-
27. J.L. Kolodner and D.B. Leake, “A tutorial introduction to ences and journals, and won the award for best technical paper at
case-based reasoning,” in Case-Based Reasoning: Experiences, the Eighteenth Annual International Conference of the British Com-
Lessons & Future Directions, edited by D.B. Leake, AAAI puter Society’s Specialist Group on Knowledge-Based Systems and
Press/MIT Press, Minlo Park, California, pp. 31–65, 1996. Applied Artificial Intelligence.