You are on page 1of 19

Appl Intell (2008) 28: 101–119

DOI 10.1007/s10489-007-0044-4

A hybrid case adaptation approach for case-based


reasoning
Claudio A. Policastro · André C.P.L.F. Carvalho ·
Alexandre C.B. Delbem

Received: 14 July 2006 / Accepted: 19 March 2007 / Published online: 18 May 2007
© Springer Science+Business Media, LLC 2007

Abstract Case-Based Reasoning is a methodology for a data set of adaptation patterns yield adaptations of the re-
problem solving based on past experiences. This methodol- trieved solutions with high predictive accuracy.
ogy tries to solve a new problem by retrieving and adapting
previously known solutions of similar problems. However, Keywords Case-based reasoning · Case adaptation ·
retrieved solutions, in general, require adaptations in order Knowledge acquisition · Hybrid systems
to be applied to new contexts. One of the major challenges
in Case-Based Reasoning is the development of an efficient
methodology for case adaptation. The most widely used 1 Introduction
form of adaptation employs hand coded adaptation rules,
Case-Based Reasoning (CBR) is a methodology for problem
which demands a significant knowledge acquisition and en-
solving based on past experiences. This methodology tries to
gineering effort. An alternative to overcome the difficulties
solve a new problem by retrieving and adapting previously
associated with the acquisition of knowledge for case adap-
known solutions of similar problems. Research in CBR is
tation has been the use of hybrid approaches and automatic directed by two main motivations [23]: The goal to model
learning algorithms for the acquisition of the knowledge human behaviors and the intent to develop new techniques
used for the adaptation. We investigate the use of hybrid ap- and construct more effective Artificial Intelligent systems.
proaches for case adaptation employing Machine Learning When CBR systems are applied for the resolution of
algorithms. The approaches investigated how to automati- problems, the retrieved solutions can rarely be directly used
cally learn adaptation knowledge from a case base and apply as a solution for a new problem. They need to be adapted in
it to adapt retrieved solutions. In order to verify the potential order to fit the requirements of the new situations. Therefore,
of the proposed approaches, they are experimentally com- case adaptation is a desirable characteristic in CBR systems.
pared with individual Machine Learning techniques. The re- Many CBR systems avoid using adaptation at all and the
sults obtained indicate the potential of these approaches as most widely used form of adaptation employs hand coded
an efficient approach for acquiring case adaptation knowl- adaptation rules, which demands a significant knowledge ac-
edge. They show that the combination of Instance-Based quisition effort for case adaptation, presenting several diffi-
Learning and Inductive Learning paradigms and the use of culties [15, 16].
An alternative to overcome the difficulties in manually
acquiring adaptation knowledge has been the use of auto-
C.A. Policastro () · A.C.P.L.F. Carvalho · A.C.B. Delbem matic learning. Until now, there has been little work related
University of São Paulo, São Paulo, Brazil to automatically learning adaptation knowledge reported in
e-mail: capoli@icmc.usp.br the literature. They can be found in [16, 24, 42, 43]. In these
A.C.P.L.F. Carvalho publications, the authors propose different approaches for
e-mail: andre@icmc.usp.br extracting adaptation knowledge from a Case Base (CB).
A.C.B. Delbem This paper extends them by investigating a new algorithm
e-mail: acbd@icmc.usp.br for automatic learning of adaptation knowledge from a CB
102 C.A. Policastro et al.

and by exploring the use of committees [9, 10, 37] of Ma- 4. Retention: the CBR system may learn the new case by
chine Learning (ML) algorithms to perform automatic case incorporating it into the CB, which is named case learn-
adaptation. ing. The fourth phase can be divided into the following
The proposed approaches, which are hybrid CBR sys- procedures: selection of relevant information to create a
tems, address the Parameter Adjustment adaptation process, new case, index composition for this case, and case in-
a strategy of substitutional adaptation [20], one of the most corporation into the CB.
frequently employed adaptation strategies for CBR systems. CBR is not a technology developed for specific purposes,
In this article, we focus on numerical solution adaptation. but rather is a general methodology of reasoning and learn-
In order to show that automatic case adaptation may re- ing [1, 18, 41], differing in important aspects from other AI
sult in considerable gains in the prediction accuracy for a so- paradigms [1]:
lution attribute, we carry out a set of experiments comparing
the performance of the proposed hybrid CBR systems with a – CBR can use specific knowledge from previous problems;
CBR system employing the K-nearestneighbor (Knn) tech- – The reasoning from previous problems is a powerful strat-
nique as a simple case adaptation mechanism. The experi- egy for problem solving, inspired by human reasoning.
ments carried out also compare the performance of the pro- The CBR paradigm is supported by two main principles
posed approach with several individual estimators generated [23]. The first principle says that the world is regular: simi-
by the same ML algorithms employed in the proposed case lar problems have similar solutions. Consequently, solutions
adaptation mechanism (CAE). Additionally, we show that from similar problems are a good starting point to solve new
further performance gains can be obtained by substituting, problems. The second principle states that problems tend to
in the case adaptation mechanism, a single ML algorithm by repeat. Therefore, new problems tend to be similar to previ-
a committee of ML algorithms. The experiments carried out ous problems.
also investigate the effect of different retrieval approaches in Cases can be represented using a number of formalisms.
the final adaptation result and the influence of the number of Depending on the attributes to be represented, a convenient
similar cases considered in the generation of adaptation pat- structure can easily be selected. The goal of a CBR sys-
terns. The empirical evaluation suggests the potential of the tem strongly influences the content of its cases [11]. A case
proposed approach by presenting experimental results where may represent different types of knowledge and assume a
it outperforms the individual ML techniques and CBR sys- large variety of representations. Additionally, a case should
tems using Knn for case adaptation. contain a certain information richness and complexity in its
This paper is organized as follows: Sect. 2 briefly intro- structure [1]. In most CBR systems the cases are usually rep-
duces the CBR paradigm. Section 3 discusses related work. resented as an attribute-value vector, divided into two parts:
Section 4 presents our hybrid system architecture and de- problem and solution. Figure 1 shows two examples for case
scribes the experiments performed. Section 5 describes the representation.
experimental results obtained. Finally, Sect. 6 presents our
conclusions.
3 Case adaptation

When CBR systems are applied to real-world problems, the


2 Case-based reasoning retrieved solutions can rarely be directly used as suitable so-
lutions for a new problem. Retrieved solutions usually re-
CBR is a methodology for problem solving based on past quire a set of adaptations in order to be applied to new con-
experiences. It tries to solve a new problem by employing a texts. An adaptation process may be either as simple as the
process of retrieval and adaptation of known solutions previ- substitution of a component (in this work, the case solution
ously applied to similar problems. CBR systems are usually attributes are named components) from the retrieved solu-
described by a reasoning cycle (also named the CBR Cycle), tion or as complex as a complete modification of the solution
which has four main phases [1]: structure. The adaptation can occur by inclusion, removal,
substitution or transformation of the components of a previ-
1. Retrieval: according to a new problem provided by the ous solution [20].
user, the CBR system retrieves, from a Case Base (CB), Several strategies for case adaptation have been proposed
previous cases similar to the new problem; in the literature [20, 35]. They can be classified in three main
2. Reuse: the CBR system adapts a solution from a retrieved groups: substitutional adaptation, transformational adapta-
case in order to fit the requirements of the new problem; tion and generative adaptation. Strategies for substitutional
3. Revision: either the CBR system or the user revises the adaptation exchange solution attribute values of the retrieved
solution generated by the reuse phase; solution with appropriate values, producing a new solution.
A hybrid case adaptation approach for case-based reasoning 103
Fig. 1 Two examples for case
representation: an
attribute-value vector and an
object-oriented case structure

Strategies for transformational adaptation modify the solu- Hanney [16] proposed an algorithm that automatically
tion structure by including or removing components of the acquires adaptation knowledge from a CB and represents
retrieved solution in order to satisfy the requirements of the this knowledge as a set of adaptation rules. When a new
new problem. Strategies for generative adaptation construct problem is presented to the CBR system, a case is retrieved
a new solution from problem data using a predefined proce- from the CB and sent to the adaptation mechanism. This
dure. When a new problem is presented, an adapted solution mechanism, in turn, extracts the differences between the
is directly obtained by applying the procedure. retrieved case and the new problem description. Next, it
Case adaptation is one of the major challenges in CBR searches in the adaptation rules set for proper rules for these
[16, 43]. Several CBR systems avoid using adaptation due differences. Finally, the adaptation mechanism generalizes
to its complexity. The most widely used form of adaptation the selected rules and applies them to the retrieved solution,
employs hand coded adaptation rules, which demands a sig- in order to obtain a new solution.
nificant effort of knowledge acquisition for case adaptation This approach employs domain-specific mechanisms due
[15, 16]. the encoding employed for the adaptation rules and the ex-
For example, in [22–24] a CBR system with a small set traction of differences between description attributes and be-
of adaptation rules and a method for memory search is pro-
tween solution component attributes. Additionally, unlike
posed. When a new problem is presented to this system, it
the approach proposed that we present in this article, it does
retrieves a similar case and sends it to the adaptation mecha-
not use existing ML algorithms for case adaptation.
nism. The adaptation mechanism, in turn, selects an adapta-
Another related work, [28], presented an adaptation
tion rule and starts a search in the memory for components
heuristic, named the difference heuristic, for case-based es-
able to substitute parts of the retrieved solution. These adap-
timation that does not rely on domain-specific rules. The
tation rules are hand coded knowledge packages acquired
approach was implemented in a CBR system in which the
specifically for a particular application domain.
Case adaptation knowledge is hard to acquire and de- concept of case dominance plays an important role in check-
mands a significant knowledge engineering effort. An alter- ing estimates based on the adaptation heuristic and in the
native to overcome these limitations in manually acquiring maintenance of consistency in the case library. The concept
adaptation knowledge has been the use of automatic learn- of case dominance is adapted from multiple-criteria deci-
ing, where case adaptation knowledge is extracted from pre- sion making, which eliminates dominated alternatives in the
viously obtained knowledge: the CB itself. Wilke and his screening of decision alternatives. To estimate the value of
colleagues [42] present a framework for learning adaptation a target case C1, the adaptation mechanism retrieves an ex-
knowledge with knowledge light approaches. This approach isting case C2 which differs from C1 only in the value of
uses knowledge already acquired and represented inside a a single case attribute, named the distinguishing attribute.
Case-Based Reasoning System. The knowledge source con- Next, the adaptation mechanism retrieves two other cases,
tainers are: vocabulary, similarity measure, the case base, C3, and C4, which also differ only in the value for the dis-
and the adaptation container. The data from the knowledge tinguishing attribute, such that C3 shares the distinguishing
containers may be pre-processed into a suitable representa- attribute value with C1 and C4 with C2. Afterwards, an es-
tion and then used by an inductive algorithm to produce the timate for the value of the target case is given by: est(C1) =
adaptation knowledge. est(C2) + est(C3) − est(C4), where est(C2), est(C3) and
104 C.A. Policastro et al.

est(C4) are the estimated values for C2, C3 and C4 in the vestigated are composed of ML algorithms, based on dif-
case library, respectively. ferent paradigms. The algorithms investigated are classical
Although this approach does not rely on domain-specific ML algorithms and there is a large number of successful
rules, it may be applied only under some circumstances or experiments with them reported in the literature [21]. Each
assumptions. algorithm (estimators) receives the same input, describing
In [43], the authors propose an inductive method for case a problem, and produces an individual output, describing a
adaptation knowledge acquisition that employs the CB as a solution. Additionally, a ML algorithm (combiner) receives
knowledge source. The method consists of an algorithm that the output of the estimators and produces the final output
executes comparisons between pairs of cases. For each pair, of the committee. The estimators and the combiner may be
it extracts the difference between the problem parts and the used to accomplish adaptations in domains with either nu-
difference between the solution parts. This algorithm uses meric or nominal components of problems and solutions.
the extracted differences of the pairs of cases, together with However, the experiments carried out in this paper employ
the description of the problem of the stored case as the cur- only domains with numeric solution attributes. The commit-
rent problem, to construct a data set of standards that will tees are composed of the following ML algorithms:
be used to train an inductive learning algorithm. The boost-
ing algorithm [14] is used to train a committee of Rise al- – Estimators—The Multi Layer Perceptron (MLP) neural
gorithms [12]—a hybrid algorithm that combines induction network [17]; the M5 regression tree learner [39]; the
and learning based on instances—using the data set gener- Support Vector Machine (SVM) technique [38];
ated. A CBR system uses the committee of Rise algorithms – Combiners—These same three ML algorithms were in-
in the phase of case adaptation. When a new problem is vestigated for the Combiner.
presented to the system, a previous case is retrieved and In every case, the committee is always composed of three
the CBR system uses the committee of algorithms to de- estimators (MLP, SVM, and M5) and one combiner (MLP,
termine what adaptation actions to perform on the retrieved SVM, or M5). Although most of the approaches of com-
case. This method is specific for design domains, due to the mittees proposed in the literature produce the final output
specific encoding employed for the adaptation training pat- by voting or simple majority [9, 10, 37], the use of a ML
terns and the extraction of differences between description algorithm as a combiner may improve the accuracy of the
attributes and between solution component attributes. system by achieving a better partition of the decision space
In another paper, [8], a Radial Basis Function (RBF) produced by the estimators. The alternatives for the investi-
neural network [31] is employed for case adaptation. When gated committees are illustrated in Fig. 2.
a new problem is presented to the CBR system, a set of simi- A MLP neural network is a common model for pattern
lar previous cases are retrieved. These cases are used to train recognition. A MLP network includes one or more hidden
the RBF network. After training, when a new problem de- layers with nonlinear activation functions that carry out suc-
scription is presented to the network, it maps the problem cessive nonlinear transformations on input patterns. Thus,
description to a solution. the intermediate layers can transform nonlinearly separable
These previous investigations explore the use inductive problems into linearly separable ones [17].
learning to acquire general adaptation knowledge from ex- M5 is a learning algorithm that generates models in the
amples and apply the acquired knowledge to automatic case form of regression trees combined with regression equations
adaptation. In this paper, we instead describe a new, domain-
(Model Tree) [39]. This model works similarly to a classi-
independent, approach for adaptation knowledge extraction
fication tree. However, the leaves contain linear expressions
and automatic case adaptation based on the induction of
instead of predicted values or classes. The Model Tree is
adaptation models by different ML algorithms.
constructed by a divide-and-conquer approach that recur-
sively creates new nodes [33].
4 Hybrid case-based reasoning system SVMs represent a family of learning algorithms based on
statistical learning theory [38]. They combine generalization
The main contribution of this paper is a new methodology control with a technique that deals with the dimensionality
for adaptation knowledge acquisition from a CB and the ex- problem.1 This technique uses hyperplanes as decision sur-
ploration of hybrid committees composed by estimators and faces and maximizes the separation margins between posi-
a combiner generated by different ML techniques to adapt tive and negative classes. In order to achieve large margins,
retrieved solutions.
1 Machine Learning algorithms can have poor performance when work-
4.1 Committee of machine learning algorithms ing on data sets with a high number of attributes. Attribute selection
techniques are frequently employed to reduce the dimensionality of
We investigates the use of a committee of ML algorithms the original data set. A SVM is able to construct a good generalization
to perform automatic case adaptation. The committees in- even for data sets with a large number of attributes.
A hybrid case adaptation approach for case-based reasoning 105
Fig. 2 Architecture of the
investigated committees

SVMs employ a statistical principle named structural risk ART2 network performs incremental clustering by creating
minimization [38]. Another key aspect of SVMs is the use of new output nodes whenever the current output nodes cannot
kernels to build support vectors from the training set. These properly represent a new example. The hybrid approach of
support vectors are instances with equal distance from the case indexing and retrieval contains two levels of memory
optimal hyperplane that divides the decision space. organization. The first level is formed by the output layer of
an ART2 network and is employed to organize clusters of
4.2 Case retrieval approach similar cases and to indicate the most similar cluster to the
problem currently presented, reducing the search space and
In this paper, to investigate the influence of the retrieval the retrieval time. The second level is a simple flat memory
mechanism on case adaptation performance, we compared that stores the instances of the cases. The cases stored in the
three different classical case retrieval approaches: one that second level (flat memory) are linked to the clusters of the
uses the KdTree structure [13], the Knn technique [27] and a first level (each case belongs to only one cluster).
hybrid mechanism [32] employing an ART2 neural network
[7] and the Knn technique. In our experiments, all these re- 4.3 Hybrid case adaptation approaches
trieval approaches use the same weights for the attributes.
Our proposed approach for case adaptation employs two
Thus, all attributes are considered to be equally important.
modules. The first module (ADAPTATION PATTERN
KdTree has been used for case retrieval in the CBR liter-
GENERATOR) produces a data set of adaptation rules. This
ature [23, 40]. It is a multi-dimensional generalization of the
data set is used by the second module (CASE ADAPTA-
one-dimension trees used for binary search, which facilitates
TION ENGINE) to train a committee of ML algorithms to
the storage and the search on groups of data of k dimensions.
automatically perform case adaptation.
This technique allows the integration of CBR with Induc-
The first module construct the adaptation patterns data
tive Learning Algorithms, providing a retrieval mechanism
set in the following way. Let x be a case stored in the CB
based on Decision Trees [4]. A KdTree recursively partitions
and yi , i = 1, . . . , k, one of the cases retrieved by the CBR
the data space into disjunct subspaces. Each node of the tree
retrieval mechanism when the problem description associ-
represents a subspace of the original data space, including
ated with x is presented. As illustrated later, the attributes in
the instances belonging to that subspace. The approach used
the problem description and solution components of x are
for the partitioning is the determination of the dimension for
concatenated to the components of the problem description
which the separation margin is maximized. The subspaces of yi to compose the input attributes of the adaptation pat-
are recursively divided until the remaining subspaces con- tern. The output attributes are the solution components of yi .
tain only one instance [4]. Next, the adaptation patterns are used to train the ML algo-
The K-nearest neighbor (Knn) algorithm [27] is the most rithms that will produce the adaptation mechanism for the
frequently used case retrieval mechanism in CBR systems second module. After training, the adaptation mechanism is
[23, 40]. Instance-based learning techniques are conceptu- used to estimate the new component value of the retrieved
ally simple algorithms. The learning process employed by solution.
Knn consists of the simple storage of the training instances This approach assumes that a CB is representative [36],
in a memory. When a new instance is presented to the sys- i.e., the CB is a good representative sample of the target
tem, a group of k similar instances is retrieved from mem- problem space and the problem space does not change over
ory. The original version of the Knn algorithm assumes that time. Therefore, no re-training of the adaptation mechanism
all stored instances correspond to points in n . For each new is required when the system creates new cases during the
instance, its closest neighbors are defined according to a dis- reasoning process.
tance metric, usually Euclidean distance, which is the metric
adopted in this paper. 4.3.1 Adaptation pattern generator
The hybrid mechanism combines the ART2 neural net-
work with the Knn technique for case memory clustering The ADAPTATION PATTERN GENERATOR module can
[32]. As a result, it speeds up the retrieval process. The extract implicit knowledge from a CB (see Fig. 3).
106 C.A. Policastro et al.
Fig. 3 The Adaptation Pattern
Generator algorithm

Initially, the function TestCaseExtract extracts a case of operations in order to illustrate time complexity calcu-
from the training set and uses it as a new problem (TestCase) lus. In block A, the main operations are the extraction of
to be presented to the CBR system. The remaining cases the problem description and solution (O(m)), and the con-
comprise a new CB without the test case (NewCB). Next, struction of the adaptation pattern (O(m)), where m is the
the function DescriptionExtract extracts, from the test case, size of a case in the case base. Therefore, block A is O(m).
the attributes of the problem (TestDescrpt) and the function Block B performs all the inner operations of block A for all
SolutionExtract extracts a component (indicated by Compo- k retrieved cases. Therefore, block B is k.O(m) = O(k.m).
nent) of the solution (TestComponent). Next, the function The main steps from block C are: (1) the extraction of a test
Retrieve, which may follow any of the retrieval approaches case from the original case base; (2) the extraction of the
presented in Sect. 4.2) returns the k most similar cases from problem description and solution of the test case; (3) the re-
the NewCB (RetrievedCases), where k is a predefined value. trieval of k similar cases from the case base; (4) operations
For each retrieved case, the attributes of the problem (RetDe- of Block B. In step 1, the algorithm accesses each case in the
scrpt) and a component of the corresponding solution (indi- order the cases are disposed in the case base, thus, each ac-
cated by Component) are extracted (RetComponent). Next, cess is O(1). Additionally, the extraction itself is O(m). As
the algorithm generates the adaptation patterns, as shown in a consequence, Step 1 is O(m). Step 2 obviously is O(m).
Fig. 5, using the function MakeAdaptationPattern, combin- Step 3 is O(x), where x is the time complexity required for
ing the following components of the cases: the retrieval mechanism (knn, kd − tree, Hybrid). Step 4 is
O(k.m). Therefore, block C is O(m) + O(m) + O(x) +
– Input attributes: the problem description stored in the test O(k.m) = O(x + k.m). Since the algorithm performs all op-
case; the problem description stored in the retrieved case; erations of the block C for all cases in the original case base
one solution component stored in the retrieved case. (see block D), it is O(n).O(x + k.m) = O(x + k.m.n) in
– Output attribute: one solution component stored in the the worst-case analysis. Thus, assuming that k and m are
previous test case, related to the solution component in constants for a CBR system, the time complexity of the
the retrieved case. algorithm is O(x + n). Therefore, the time complexity of
our algorithm is limited by the adopted retrieval mechanism
Finally, the adaptation data sets produced are used to train
(O(n) for Knn, and O(log n) for KdTree, for search and in-
the committee of ML algorithms, which are employed as
sertion [29]).
adaptation mechanisms. Figure 5 shows an example of an adaptation pattern pro-
In the experiments carried out for each pattern adaptation duced using one of the data sets employed in this article and
data set, initially, each of the three techniques (MLP, SVM the training of the hybrid committee. This example shows
and M5) is individually trained. The trained techniques be- one adaptation pattern generated for the AG1 solution com-
come the estimators of the committees. Next, for each in- ponent of the European data set.
stance, the output values produced by the estimators are
combined, creating a data set to be used by the combiners. 4.3.2 Case adaptation engine
The adaptation mechanisms adapt the value of just one out-
put attribute (a single solution component). Therefore, one The case adaptation approaches investigated in this paper
independent pattern adaptation data set must be produced learn the modifications that need to be performed to the val-
for each solution component. This strategy preserves the in- ues of the components of a retrieved solution in order to
dependence of the approach from the structure of the cases. achieve an adequate solution for a new problem. The most
Figure 4 shows the algorithm for the ADAPTATION important feature of the adaptation mechanism is the em-
PATTERN GENERATOR, which is organized into blocks ployment of implicit knowledge obtained from the CB with
A hybrid case adaptation approach for case-based reasoning 107
Fig. 4 ADAPTATION
PATTERN GENERATOR
algorithm. The blocks A, B, C,
and D represent blocks of
operations

a minimum knowledge acquisition effort. The case adapta- M5 algorithm has complexity O(n2 log n) [26]. The retrieval
tion process is described in Fig. 6. techniques also have different complexities, which are O(n)
When a new problem is presented to the CBR system, for Knn, and O(log n) for KdTree, for search and insertion
the most similar case stored in the CB is retrieved by the re- [29].
trieval mechanism. A retrieved case (RetrievedCase) is sent Therefore, the main operations of this algorithm are: the
to the adaptation engine together with the new problem de- operations on the attributes of the retrieved case: O(m), the
scription (Description). The adaptation engine, in turn, ex- operations on the attributes of the new problem description:
tracts the attributes from the new problem description (Ret- O(m), and the adaptation of the attribute solution O(x).
Description). Next, the adaptation engine extracts the so- Therefore, the algorithm takes O(x) in the worst case, where
lution component (RetSolution), indicated by Component, its computational complexity is proportional to the complex-
and the function MakeInputPattern generates an adaptation ity of the ML algorithms employed in the hybrid committee
pattern for this component. Next, the function Adaptation- and the search techniques.
Mechanism computes the new value of the component of the Figure 7 shows an example of the problem solving
retrieved solution (NewComponent), which is returned to the process employing a trained committee. The example is re-
system. When the case solution has more than one compo- lated to the AG1 component of the European data set.
nent (like in the European data set), the adaptation engine is Figure 8 shows the general architecture of the proposed
called for each component. As a result, a set of values is re- hybrid system, highlighting the role of the algorithms in the
turned and composed to create the complete solution of the CBR Cycle.
new problem.
The main operations of this algorithm are the extraction
of the attributes from the retrieved case, the construction of 5 Evaluation
the input pattern with the extracted attributes, and the sub-
mission of the input pattern to the adaptation mechanism This section presents the main results from a set of experi-
(committee of ML techniques). ments carried out to evaluate the performance of automatic
The algorithm receives as input the retrieved case, to- case adaptation. First, the hybrid CBR systems using indi-
gether with the description and the solution of the new prob- vidual ML techniques are empirically compared with indi-
lem. It extracts each attribute from the problem description vidual estimators produced by ML techniques. Next, in or-
and solution of the retrieved case, taking O(m) for these der to show that committees of ML algorithms may lead to
operations, where m is the size of a case in the case base. a higher prediction accuracy, we compare hybrid case adap-
Next, the algorithm combines the attributes to form the input tation approaches, using committees of ML techniques and
pattern, taking O(m) for this operation. Then the algorithm individual ML techniques, with individual ML techniques.
submits the input pattern to the adaptation mechanism, tak- The hybrid approaches are also compared with a stan-
ing O(x) for this operation, where x is the computational dard CBR system employing Knn for case adaptation. In
complexity of the ML estimators used in the committee. this standard approach, the value of the solution component
Each estimator has a different complexity, which is influ- is obtained by averaging the solution of the k most similar
enced by the parameters optimized. For example, SVMs cases retrieved by the Knn technique. Finally, the experi-
have complexity O(n3 ) [6], training of MLP networks is ments investigate the effect of using different retrieval ap-
O(MW) for each epoch [34] (where M is the number of in- proaches and of varying the number of similar cases recov-
stances and W is the number of weights), and training the ered in the final adaptation result.
108 C.A. Policastro et al.

Fig. 5 Example of adaptation pattern generation

Fig. 6 The CASE


ADAPTATION ENGINE
algorithm

The experimental tests followed the 10-fold-cross- was used as a test fold (a set of new problems to be presented
validation methodology [27]. The patterns were randomly to the system) and the remaining 9 folds made up a training
divided into 10 groups (folds) with similar size. One fold fold (a set of previously stored cases). For each one of the
A hybrid case adaptation approach for case-based reasoning 109

Fig. 7 Example of the problem solving process

10 folds, the training-fold (using the remaining 9 folds) was adaptation patterns were independently produced for each
used to train the ML algorithms employed alone and to gen- fold.
erate the adaptation patterns used to train the proposed case
adaptation mechanism of the CBR systems. Afterwards, the 5.1 Experiments setup
average absolute error rates (the absolute difference between
the expected and the produced output value) were calculated As previously mentioned, estimators and combiners were
using the test fold. Based on the error rates for the 10 folds, generated by three ML techniques: MLP, M5, and SVM.
their average and standard deviation for all folds were cal- The topologies of the estimators generated by the MLP net-
culated. work are shown in Table 1.
In the experiments, the prediction performances of the The MLP network and the M5 algorithm were simu-
individual ML estimators (MLP, M5, and SVM) trained with lated using the WEKA library, version 3.2—which includes
the original data sets were compared with those obtained by a set of Machine Learning algorithms.2 The SVMs were
the CBR systems, where each individual ML estimator was simulated using the LIBSVM tool.3 The MLP networks
trained using the adaptation patterns, and the CBR system
using Knn. The training adaptation patterns were produced
2 Available at http://www.cs.waikato.ac.nz/ml/weka/index.htm.
from the same folds of the original training set, using the
3 Available
CBR retrieval mechanism. Thus, for the comparisons, the at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
110 C.A. Policastro et al.
Fig. 8 General architecture of
the CBR Cycle with the
proposed case adaptation
approach. APG is the
Adaptation Pattern Generator
Algorithm (see Fig. 3) and CAE
is the CASE ADAPTATION
ENGINE Algorithm (see Fig. 6)

Table 1 Topology of the estimators generated by the MLP network CBR(RA − AA − k). In this notation, RA indicates the ap-
Data set Input Hidden layer Output proach used for case retrieval (KdTree, Knn or Hybrid), AA
indicates the approach used for case adaptation (MLP, M5,
Servo 25 13 1 SVM, CMLP, CM5. or CSVM), and k indicates the number
Imports 162 82 1 of cases similar to the test case considered during adaptation
Housing 27 14 1 patterns generation. For example, CBR(KdTree − CSVM −
Michigan 27 14 1 3) means a CBR system employing the KdTree approach for
European 37 20 1 case retrieval and a committee using the SVM technique as
a combiner for case adaptation, trained with adaptation pat-
terns generated using the 3 cases most similar to the test
were trained using the momentum backpropagation algo-
case. Additionally, the use of ∗ means any mechanism or
rithm [17] with the moment term set to 0.2 and learning rate
approach. For the standard CBR system employing the Knn
set to 0.3. The M5 algorithm was trained using the default
average value as the adaptation mechanism, the employed
parameters settings of the WEKA Library. The SVMs were
notation is the following: CBR(Knn − k), where k indicates
trained using the Radial Basis Function kernel and default
the number of similar cases considered.
parameters settings of the LIBSVM tool.
The experiments carried out employ five data sets from
different domains in order to show the generality of the pro- 5.2 Empirical results
posed approach. For a detailed description of the structure
of these data sets, see Appendix 1. Three different adapta-
This section presents the main results obtained from our ex-
tion pattern data sets were created by generating adaptation
periments. Table 2 shows the main results of the experiments
patterns using the k most similar cases, k = {1, 3, and 5}
(absolute error ± standard deviation) for the individual tech-
(see Sect. 4.3.1). The data preprocessing was performed
only when required by the retrieval and adaptation mech- niques employed alone and for the CBR systems employing
anisms. The cases were stored in the CB in their original automatic case adaptation approaches for the three different
format. The numeric values were normalized to the inter- retrieval methods (see Sect. 4.2) using different values of k,
val [0 . . . 1] using the minimum and the maximum values for each data set. Since the European data set has more than
already present in the CB (we are supposing that the CB is one solution, the average absolute error of all components
representative [36]). The input attributes with nominal val- was calculated for this data set. For the complete set of re-
ues were transformed into orthogonal vectors of binary val- sults of the experiments carried out, see Appendix 2.
ues. A first look at these results suggests the superiority of
The results reported in this paper employ the follow- the hybrid approach. In order to verify the statistical signif-
ing notation for the tests with the hybrid CBR models: icance of this superiority, we ran the two-tailed paired t test
A hybrid case adaptation approach for case-based reasoning 111
Table 2 Main results of the experiments 6 Discussion
Model Average absolute error
The experimental results presented in the previous section
Servo data set show that using committees of ML techniques as adapta-
CBR(Hybrid-M5-5) 0.024 ± 0.019 tion mechanisms generally reduces the system’s average ab-
CBR(Hybrid-CMLP-5) 0.024 ± 0.011 solute prediction error and its standard deviation. These re-
CBR(Knn-1) 0.552 ± 0.278 sults possibly occur due to the fact that the committee is
CSVM 0.532 ± 0.299 able to benefit from the different biases of the estimators.
Therefore, these results show the potential of using hybrid
Imports data set
approaches that combine inductive learning and instance-
CBR(Knn-CM5-3) $518.85 ± $83.05
based learning.
CBR(Knn-1) $1.323.56 ± 329.26
The hybrid committee of ML algorithms takes advan-
CSVM $1.051.31 ± $311.53 tage of using different techniques with different inductive
Housing data set biases for the search in the decision space. Also the use of
CBR(Knn-CSVM-5) 2.37 ± 0.46 a combiner usually leads to a better partition of the search
CBR(Knn-1) 3.16 ± 0.83 space.
CSVM 3.02 ± 0.12 These results suggest that the adaptation pattern data
set extracted from the CB contains a good representative
Michigan data set sample of the required adaptations for the solution compo-
CBR(Knn-CSVM-3) 4.51 ± 0.38 nents.
CBR(KdTree-CSVM-5) 4.49 ± 0.38 Finally, the results illustrate that the case retrieval mech-
CBR(Knn-1) 6.02 ± 1.74 anism used by the system can influence the final results
CSVM 7.98 ± 0.42 of the hybrid CBR system, indicating that using a more
accurate retrieval mechanisms can increase system accu-
European data set
racy.
CBR(Hybrid-CSVM-3) 2.76 ± 0.19
CBR(Hybrid-CSVM-5) 2.77 ± 0.09
CBR(Knn-1) 6.11 ± 1.58
7 Conclusions
CSVM 3.50 ± 0.22

This article investigated the use of a hybrid case adaptation


mechanism able to perform automatic case adaptation on
several domains. Our results suggest that the use of a hy-
brid committee of ML techniques increased prediction ac-
with 99% certainty [27, 30]. The statistical results are shown curacy.
in Table 3. According to the results, the CBR(∗ − CSVM − ∗) mod-
In contrast to the work presented by Hanney [16] and els produced, in general, the best prediction results. This is
Wiratunga et al. [43], we developed a generic automatic case probably due to the properties of the SVM algorithm. SVMs
adaptation approach that may be used on domains that re- use a generalization control (inductive bias) and kernel func-
tions that allow the construction of hyperplanes that create
quire parameter adaptation. The knowledge extraction algo-
more efficient decision surfaces.
rithm we proposed employs a simple encoding for adapta- The results indicate the potential of the hybrid approach.
tion patterns. This is achieved by concatenating components Our analysis of the results also reveals that increasing
of the problem description and solution, instead of extract- the number of similar cases considered during the gen-
ing attribute differences and adjustments. eration of the adaptation patterns may influence the pre-
The number (k) of similar cases used to generate the diction performance of case adaptation. Further investiga-
adaptation pattern mechanism may influence adaptation per- tions are required to determine more precisely the influence
formance. For large data sets and/or data sets with a large of k.
Additionally, according to the results, in general, com-
number of continuous input attributes, an increase of k may
mittees of ML algorithms increased system accuracy. The
decrease the performance of the system and k = 1 is more average error rates and the standard deviations were, in gen-
appropriate. This may be due to the increase in the vari- eral, smaller than those obtained for hybrid CBR systems
ability and conflicting adaptation rules in the adaptation pat- using the ML algorithms individually for the case adapta-
terns. tion.
112 C.A. Policastro et al.
Table 3 T test conclusions
Model Conclusion

Servo data set


CBR(Hybrid - M5 - 5) and CBR(Hybrid - CMLP - 5) similar performance
CBR(Hybrid - CMLP - 5) and CMLP CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CM5 CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CSVM CBR(Hybrid - CMLP - 5) is better
CBR(Hybrid - CMLP - 5) and CBR(Knn - 1) CBR(Hybrid - CMLP - 5) is better

Imports data set


CBR(Knn - CM5 - 3) and CM5 CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CMLP CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CSVM CBR(Knn - CM5 - 3) is better
CBR(Knn - CM5 - 3) and CBR(Knn - 1) CBR(Knn - CM5 - 3) is better

Housing data set


CBR(Knn - CSVM - 5) and CM5 CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CMLP CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CSVM CBR(Knn - CSVM - 5) is better
CBR(Knn - CSVM - 5) and CBR(Knn - 1) CBR(Knn - CSVM - 5) is better

Michigan data set


CBR(Knn - CSVM - 5) and CBR(KdTree - CSVM - 5) similar performance
CBR(KdTree - CSVM - 5) and CMLP CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CM5 CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CSVM CBR(KdTree - CSVM - 5) is better
CBR(KdTree - CSVM - 5) and CBR(Knn - 1) CBR(KdTree - CSVM - 5) is better

European data set


CBR(Hybrid - CSVM - 3) and CBR(Hybrid - CSVM - 5) similar performance
CBR(Hybrid - CSVM - 3) and CMLP CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CM5 CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CSVM CBR(Hybrid - CSVM - 3) is better
CBR(Hybrid - CSVM - 3) and CBR(Knn - 1) CBR(Hybrid - CSVM - 3) is better

The experiments with the European data set indicated order to eliminate noisy, distant, conflicting, or duplicated
that the case adaptation approach can be easily extended to patterns. We will also investigate the influence of differ-
domains where the solution of the cases has more than one ent similarity functions and attributes weights during case
component, by creating one independent adaptation data set retrieval. Additionally, we believe that the investigation of
and one independent ML algorithm or committee for each committees composed of a higher number of estimators can
component of the solution, handling these components as further decrease error rates. Finally, we will compare the
distinct problems. proposed approaches with additional CBR and case adap-
Our approach employs a process of adaptation pattern tation approaches. Another future work task will be to com-
generation that can reduce knowledge acquisition effort. pare our approach versus using Locally Weighted Learning
Moreover, the process employed to obtain an adaptation pat- [2] for case adaptation.
tern data set is fully integrated with the case retrieval mech-
Acknowledgement The authors would like to thank CNPq and
anism and can benefit from standard retrieval techniques [4, FAPESP for support received.
13, 25]. Finally, the results obtained suggest that the set of
adaptation rules extracted from the CB used can provide a
useful new approach for case adaptation, with a good poten- Appendix 1
tial to solve problems.
For future work, we plan to investigate the influence of The hybrid CBR system described in this paper was eval-
post-processing the generated adaptation patterns data set in uated using different data sets. These evaluations involved
A hybrid case adaptation approach for case-based reasoning 113

a large number of experiments, which analysed the poten- screw/nut, and a sliding carriage. The output value rep-
tial of the combination of ML algorithms for automatic resents the time required for the system to respond to
case adaptation. This appendix presents the main features a step change in a position set point. This data set has
of the five data sets employed in this paper are presented. been used by Hanney [16] to evaluate another case adap-
These data sets, except for the Michigan and European tation strategy. For a summary of this data set, see Ta-
data sets, come from the UCI Repository of ML Data- ble 4);
bases [5]. Imports This data set lists the predicted price of a car given
Servo This data set was extracted from the simulation of a a set of specifications (see Table 5);
servo system involving a servo amplifier, a motor, a lead Housing The Housing data set contains housing values in
suburbs of Boston. This data set can be used for the predic-
Table 4 Servo case structure tion of the median value of owner-occupied homes, given
the value of a set of attributes (see Table 6);
Attribute Values
Michigan This data set comes from a water quality study
Problem Motor A,B,C,D,E carried out in Michigan, USA [19]. The instances in this
Screw A,B,C,D,E data set were analysed for several physical and chemical
PGain 3,4,5,6 parameters, such as: depth, secch, color, turbidity, conduc-
VGain 1,2,3,4,5 tivity and pH. For each case, the first 13 attributes represent
physical and chemical measurements, considered relevant
Solution Class Continuous
to predict plankton concentration. The last value of each

Table 5 Imports case structure


Attribute Value

Problem risk −3, −2, −1, 0, 1, 2, 3


loss-normalize Continuous
manufacturer alfa-romero, audi, bmw, chevrolet, dodge, ...
fuel-type diesel, gas
aspirantion std, turbo
door-number four, two
cover-type hardtop, wagon, sedan, hatchback, convertible
traction 4wd, fwd, rwd
motor-location front, rear
wheelwork-relationship Continuous
width Continuous
length Continuous
height Continuous
weight-limit Continuous
motor-type dohc, dohcv, l, ohc, ohcf, ohcv, rotor
cylinder-number eight, five, four, six, three, twelve, two
motor-size Continuous
fuel-system 1bbl, 2bbl, 4bbl, idi, mfi, mpfi, spdi, spfi
cylinder-capacity Continuous
course Continuous
tax-compression Continuous
potency Continuous
maximum-rotation Continuous
consumption-city Continuous
consumption-highway Continuous

Solution Price Continuous


114 C.A. Policastro et al.

case represents the concentration of planktons in the water ples were taken from sites on different European rivers over
(see Table 7); a period of approximately one year. The instances were an-
European This data set was employed in the 1999 Compu- alyzed for various chemical substances, including: nitrogen
tational Intelligence and Learning (COIL) competition. It in the form of nitrates and ammonia, phosphate, pH, oxy-
can be downloaded from the UCI KDD Repository [3]. gen and chloride. In parallel, algae samples were collected
This data set comes from a water quality study where sam- to determine the algae population distributions. The first 11
attributes of each case represent the season, the river size,
Table 6 Housing case structure the fluid velocity and 8 chemical concentrations, consid-
ered relevant to predict the algae population distributions.
Attribute Value
The last 7 values of each case are the distribution of differ-
ent types of algae (see Table 8).
Problem per-capta-crime-tax Continuous
residential-lands-proportion Continuous Table 9 summarizes the main characteristics of these data
unnegotiated-land-proportion Continuous sets. The column  Instances shows the number of instances
charles-river-variable Continuous in each data set. The column  Attributes (Continuous, Sym-
nitric oxide concentration Continuous bolic) presents the total number of attributes, and the number
home-rooms-medium-number Continuous of numeric and nominal attributes. The column  Missing
owner-occupied-units-proportion Continuous Attributes lists the total number of missing attributes in the
five-centers-boston-ponderate-distance Continuous original data set. The column  Conflicting and Duplicated
radial-highways-accessibility-index Continuous Attributes shows the total number of conflicting instances
tax-rates-over-properties Continuous plus the total number of duplicated instances (the number of
learning-taxes Continuous cases with the same problem description, but different solu-
negro-proportion Continuous
tions and cases with the same problem description and the
same solution).
% low-income-population Continuous

Solution home-medium-value Continuous


Appendix 2

This appendix presents complete set of results obtained in


Table 7 Michigan case structure
the experiments carried out for this paper.
Attribute Value

Problem Depth, ft Continuous


Secchi, m Continuous Table 8 European case structure
Color Continuous Attribute Value
Turbidity Top Continuous
Turbidity Bot Continuous Problem Season summer, spring, autumn, winter
Conductivity Top Continuous Size small, medium, large
pH Top Continuous Velocity high, medium, low
pH Botton Continuous CC1 Continuous
Temp C Top Continuous ... ...
Temp C Botton Continuous CC8 Continuous
Dissolved Oxygen Top Continuous
Solution AG1 Continuous
Dissolved Oxygen Botton Continuous
... ...
Solution Plankton Continuous AG7 Continuous

Table 9 Data set characteristics


summary Data set Instances Attributes Attributes Duplicated or Solution
(Numeric, Nominal) Missing Conflicting Components

Servo 167 4(0,4) 0 0 1


A hybrid case adaptation approach for case-based reasoning 115
Table 10 Results of CBR
Systems for the Servo data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) 0.032 ± 0.017 0.027 ± 0.018 0.024 ± 0.018


CBR(Knn-M5) 0.032 ± 0.018 0.028 ± 0.019 0.026 ± 0.015
CBR(Hybrid-M5) 0.032 ± 0.017 0.026 ± 0.018 0.024 ± 0.019
CBR(KdTree-MLP) 0.041 ± 0.021 0.033 ± 0.020 0.032 ± 0.017
CBR(Knn-MLP) 0.053 ± 0.022 0.034 ± 0.019 0.028 ± 0.015
CBR(Hybrid-MLP) 0.033 ± 0.020 0.033 ± 0.020 0.034 ± 0.019
CBR(KdTree-SVM) 0.067 ± 0.014 0.076 ± 0.009 0.075 ± 0.009
CBR(Knn-SVM) 0.050 ± 0.020 0.076 ± 0.010 0.075 ± 0.009
CBR(Hybrid-SVM) 0.078 ± 0.010 0.077 ± 0.009 0.075 ± 0.010
CBR(KdTree-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(Knn-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(Hybrid-CM5) 0.032 ± 0.019 0.032 ± 0.018 0.032 ± 0.019
CBR(KdTree-CMLP) 0.038 ± 0.019 0.030 ± 0.019 0.025 ± 0.012
CBR(Knn-CMLP) 0.045 ± 0.020 0.028 ± 0.017 0.026 ± 0.014
CBR(Hybrid-CMLP) 0.033 ± 0.019 0.032 ± 0.020 0.024 ± 0.011
CBR(KdTree-CSVM) 0.039 ± 0.015 0.038 ± 0.016 0.040 ± 0.017
CBR(Knn-CSVM) 0.035 ± 0.015 0.039 ± 0.017 0.036 ± 0.015
CBR(Hybrid-CSVM) 0.042 ± 0.014 0.038 ± 0.015 0.043 ± 0.019
CBR(Knn) 0.55 ± 0.28 1.14 ± 0.30 1.24 ± 0.31

Table 11 Results of CBR


Systems for the Imports data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) $1.143.78 ± $278.49 $612.02 ± $78.22 $652.08 ± $76.67


CBR(Knn-M5) $1.127.69 ± $282.66 $614.06 ± $79.31 $652.06 ± $84.73
CBR(Hybrid-M5) $1.147.47 ± $275.31 $612.11 ± $81.29 $659.09 ± $74.51
CBR(KdTree-MLP) $1.882.84 ± $539.62 $1.129.41 ± $133.72 $992.35 ± $121.62
CBR(Knn-MLP) $1.940.78 ± $653.71 $1.093.41 ± $140.41 $103.31 ± $133.67
CBR(Hybrid-MLP) $1.841.76 ± $467.78 $1.044.36 ± $135.25 $988.35 ± $117.59
CBR(KdTree-SVM) $1.412.41 ± $226.41 $1.088.36 ± $109.33 $1.084.34 ± $71.87
CBR(Knn-SVM) $1.420.33 ± $234.26 $1.079.35 ± $96.63 $1.089.38 ± $68.72
CBR(Hybrid-SVM) $1.398.33 ± $221.79 $1.089.35 ± $111.34 $1.079.35 ± $69.67
CBR(KdTree-CM5) $1.018.42 ± $204.21 $577.85 ± $83.05 $731.07 ± $90.43
CBR(Knn-CM5) $1.029.42 ± $217.58 $518.85 ± $83.05 $738.07 ± $96.28
CBR(Hybrid-CM5) $1.008.42 ± $1987.30 $579.85 ± $85.05 $731.07 ± $88.23
CBR(KdTree-CMLP) $1.167.03 ± $178.45 $639.21 ± $85.30 $699.21 ± $94.94
CBR(Knn-CMLP) $1.160.05 ± $178.17 $627.19 ± $75.36 $688.19 ± $89.59
CBR(Hybrid-CMLP) $1.168.03 ± $171.41 $637.21 ± $87.93 $689.21 ± $94.77
CBR(KdTree-CSVM) $1.102.04 ± $124.09 $623.21 ± $69.77 $632.16 ± $61.12
CBR(Knn-CSVM) $1.109.04 ± $139.82 $618.23 ± $72.28 $639.17 ± $53.38
CBR(Hybrid-CSVM) $1.089.15 ± $116.18 $629.15 ± $66.05 $627.14 ± $63.10
CBR(Knn) 1.323.56 ± 329.26 1.582.36 ± 609.37 1.678.14 ± 499.11
116 C.A. Policastro et al.
Table 12 Results of CBR
Systems for the Housing data set Model Average absolute error
k=1 k=3 k=5

CBR(KdTree-M5) 3.44 ± 0.28 3.68 ± 0.45 3.93 ± 0.52


CBR(Knn-M5) 3.45 ± 0.28 3.78 ± 0.51 4.10 ± 0.64
CBR(Hybrid-M5) 3.43 ± 0.29 3.62 ± 0.41 3.82 ± 0.45
CBR(KdTree-MLP) 4.28 ± 0.61 3.98 ± 0.53 4.26 ± 0.48
CBR(Knn-MLP) 3.94 ± 0.52 3.96 ± 0.46 4.00 ± 0.38
CBR(Hybrid-MLP) 4.51 ± 0.66 4.00 ± 0.58 4.44 ± 0.54
CBR(KdTree-SVM) 3.45 ± 0.36 3.18 ± 0.30 3.12 ± 0.29
CBR(Knn-SVM) 3.46 ± 0.38 3.17 ± 0.31 3.11 ± 0.29
CBR(Hybrid-SVM) 3.45 ± 0.35 3.19 ± 0.29 3.12 ± 0.28
CBR(KdTree-CM5) 3.65 ± 0.42 3.54 ± 0.47 4.23 ± 1.06
CBR(Knn-CM5) 3.47 ± 0.36 3.53 ± 0.40 3.95 ± 0.90
CBR(Hybrid-CM5) 3.77 ± 0.45 3.55 ± 0.51 4.42 ± 1.17
CBR(KdTree-CMLP) 3.82 ± 0.38 3.53 ± 0.41 3.77 ± 0.42
CBR(Knn-CMLP) 3.76 ± 0.43 3.67 ± 0.36 3.63 ± 0.34
CBR(Hybrid-CMLP) 3.85 ± 0.35 3.44 ± 0.45 3.87 ± 0.48
CBR(KdTree-CSVM) 2.89 ± 0.72 2.55 ± 0.21 2.52 ± 0.29
CBR(Knn-CSVM) 3.07 ± 1.63 2.44 ± 0.35 2.37 ± 0.46
CBR(Hybrid-CSVM) 2.78 ± 0.12 2.63 ± 0.12 2.62 ± 0.17
CBR(Knn) 3.16 ± 0.83 3.65 ± 2.35 3.28 ± 2.27

Table 13 Results of CBR


Systems for the Michigan data Model Average absolute error
set k=1 k=3 k=5

CBR(KdTree-M5) 12.35 ± 2.09 14.42 ± 3.58 16.12 ± 3.53


CBR(Knn-M5) 12.53 ± 2.47 14.68 ± 4.06 16.33 ± 3.72
CBR(Hybrid-M5) 12.19 ± 2.89 15.15 ± 3.27 16.49 ± 3.59
CBR(KdTree-MLP) 22.98 ± 3.98 23.33 ± 5.31 23.97 ± 7.45
CBR(Knn-MLP) 23.25 ± 4.27 23.70 ± 6.02 24.20 ± 7.54
CBR(Hybrid-MLP) 24.78 ± 4.33 26.10 ± 5.58 24.66 ± 5.02
CBR(KdTree-SVM) 11.51 ± 0.61 12.98 ± 1.51 12.99 ± 1.72
CBR(Knn-SVM) 13.10 ± 1.37 13.16 ± 1.79 13.15 ± 1.87
CBR(Hybrid-SVM) 12.18 ± 2.85 13.10 ± 2.34 12.87 ± 2.16
CBR(KdTree-CM5) 20.97 ± 4.40 21.30 ± 6.24 23.10 ± 5.99
CBR(Knn-CM5) 21.29 ± 4.81 21.67 ± 7.12 23.36 ± 6.02
CBR(Hybrid-CM5) 22.21 ± 4.58 24.10 ± 5.78 24.72 ± 5.58
CBR(KdTree-CMLP) 20.98 ± 4.40 21.31 ± 6.33 23.11 ± 5.88
CBR(Knn-CMLP) 21.24 ± 4.81 21.69 ± 7.00 23.33 ± 6.20
CBR(Hybrid-CMLP) 20.09 ± 4.13 23.31 ± 5.60 24.63 ± 4.87
CBR(KdTree-CSVM) 6.43 ± 0.50 4.78 ± 0.30 4.49 ± 0.38
CBR(Knn-CSVM) 6.51 ± 0.70 4.85 ± 0.36 4.51 ± 0.38
CBR(Hybrid-CSVM) 5.78 ± 0.59 4.79 ± 0.29 4.68 ± 0.23
CBR(Knn) 6.02 ± 1.74 6.11 ± 1.90 6.40 ± 2.02
A hybrid case adaptation approach for case-based reasoning 117
Table 14 Prediction results of
the CBR Systems for the Model Average absolute error
European data set k=1 k=3 k=5

CBR(KdTree-M5) 4.45 ± 1.12 5.10 ± 1.60 5.83 ± 2.63


CBR(Knn-M5) 4.60 ± 1.15 5.12 ± 1.75 5.80 ± 3.05
CBR(Hybrid-M5) 5.31 ± 0.76 5.48 ± 0.57 5.99 ± 0.87
CBR(KdTree-MLP) 10.82 ± 2.77 9.75 ± 1.98 9.98 ± 1.93
CBR(Knn-MLP) 10.91 ± 2.70 9.97 ± 2.24 10.43 ± 2.54
CBR(Hybrid-MLP) 11.54 ± 0.32 10.24 ± 0.88 11.76 ± 0.80
CBR(KdTree-SVM) 5.55 ± 0.38 5.34 ± 0.33 5.31 ± 0.43
CBR(Knn-SVM) 5.61 ± 0.45 5.54 ± 0.49 5.65 ± 0.50
CBR(Hybrid-SVM) 5.90 ± 0.50 5.80 ± 0.40 5.87 ± 0.36
CBR(KdTree-CM5) 7.99 ± 2.55 7.24 ± 1.77 6.91 ± 2.02
CBR(Knn-CM5) 8.21 ± 2.79 7.52 ± 2.03 7.38 ± 2.51
CBR(Hybrid-CM5) 8.64 ± 0.42 7.84 ± 0.60 8.11 ± 0.69
CBR(KdTree-CMLP) 8.33 ± 2.47 8.11 ± 1.80 8.41 ± 2.16
CBR(Knn-CMLP) 8.57 ± 2.70 8.33 ± 2.11 8.71 ± 2.40
CBR(Hybrid-CMLP) 8.82 ± 0.50 8.70 ± 0.70 8.85 ± 0.56
CBR(KdTree-CSVM) 2.96 ± 0.32 3.49 ± 0.29 3.11 ± 0.22
CBR(Knn-CSVM) 3.07 ± 0.40 3.56 ± 0.36 3.23 ± 0.30
CBR(Hybrid-CSVM) 2.99 ± 0.03 2.76 ± 0.19 2.77 ± 0.09
CBR(Knn) 6.11 ± 1.58 6.13 ± 1.20 6.25 ± 1.19

Table 15 Results of ML Techniques for the Servo data set Table 17 Results of ML Techniques for the Housing data set

Model Average absolute error Model Average absolute error

M5 0.959 ± 0.431 M5 3.43 ± 0.29


MLP 0.976 ± 0.302 MLP 4.28 ± 0.94
SVM 0.832 ± 0.319 SVM 3.18 ± 0.33
CM5 0.559 ± 0.311 CM5 3.17 ± 0.22
CMLP 0.586 ± 0.202 CMLP 3.95 ± 0.34
CSVM 0.532 ± 0.299 CSVM 3.02 ± 0.12

Table 16 Results of ML Techniques for the Imports data set Table 18 Results of ML Techniques for the Michigan data set

Model Average absolute error Model Average absolute error

M5 $1.583.66 ± $321.32 M5 12.34 ± 2.05


MLP $1.751.31 ± $483.52 MLP 12.81 ± 1.45
SVM $1.651.31 ± $471.58 SVM 24.88 ± 6.59
CM5 $1.283.66 ± $221.31 CM5 11.91 ± 1.34
CMLP $1.251.31 ± $363.53 CMLP 12.74 ± 2.03
CSVM $1.051.31 ± $311.53 CSVM 7.98 ± 0.42
118 C.A. Policastro et al.
Table 19 Prediction results of the ML Techniques for the European 19. Jackson WG (2002) Water resources outreach education program
data set for schools and community groups. Annis Water Resources Insti-
Model Average absolute error tute. http://www.gvsu.edu/wri/education/waterdata.htm
20. Kolodner J Adaptation methods and strategies. Case-based reason-
ing. Kaufmann, San Francisco, Chap. 11
M5 5.17 ± 0.54
21. Lavrac N, Gamberger D, Todorovski L, Blockeel H (eds) (2003)
MLP 11.14 ± 0.96 Machine learning: ECML 2003. In: Proceedings of the 14th Eu-
SVM 5.74 ± 0.46 ropean conference on machine learning, Cavtat-Dubrovnik, Croa-
CM5 4.68 ± 0.31 tia. Lecture notes in computer science, vol 2837. Springer, Berlin,
ISBN 3-540-20121-1
CMLP 9.50 ± 0.63
22. Leake D (1995) Becoming an expert case-based reasoner: learning
CSVM 3.50 ± 0.22 to adapt prior cases. In: 8th annual Florida artificial intelligence
research symposium, Melbourne, pp 1120–1160
23. Leake D (1996) CBR in context: the present and future. In:
References Case-based reasoning: experiences, lessons and future directions.
AAAI/MIT, Menlo Park, pp 1–35
1. Aamodt A, Plaza E (1994) Case-based reasoning: foundational 24. Leake D, Kinley A, Wilson D (1996) Acquiring case adaptation
issues, methodological variations, and systems approaches. AI knowledge: a hybrid approach. In: Burkhard D, Lenz M (eds) 30th
Commun 7:39–59 national conference on artificial intelligence and 8th innovative
2. Atkeson CG, Moore AW, Schaal S (1997) Locally weighted learn- applications of artificial intelligence conference, Portland, USA.
ing. Artif Intell Rev 11:11–73 AAAI/MIT, Menlo Park, pp 684–689
3. Bailey T, Elkan C (1993) Estimating the accuracy of learned con- 25. Lenz M, Burkhard H-D (1996) Case retrieval nets: basic ideas and
cepts. In: Bajcsy R (ed) 13th international joint conference on ar- extensions. In: Burkhard H-D, Lenz M (eds) 4th German work-
tificial intelligence, Chambry, France. Kaufmann, San Francisco, shop on case-based reasoning: system development and evalua-
pp 895–901 tion, Berlin, Germany, pp 103–110
4. Bentley J (1975) Multidimensional binary search tree used for as- 26. Malerba D, Appice A, Bellino A, Ceci M, Pallotta D (2001) Step-
sociative searching. Commun ACM 18(9):509–517 wise induction of model trees. In: Esposito F (ed) AI*IA 2001:
5. Blake CL, Merz CJ (1998) UCI repository of machine learn- advances in artificial intelligence. Lecture notes in artificial intel-
ing databases. University of California, Irvine, Dept. of Infor- ligence, vol 2175. Springer, New York
mation and Computer Sciences. http://www.ics.uci.edu/~mlearn/ 27. Mitchell TM (1997) Machine learning. McGraw–Hill, New
MLRepository.html York
6. Burges CJC (1998) A tutorial on support vector machines for pat- 28. McSherry D (1998) An adaptation heuristic for case-based estima-
tern recognition. Data Min Knowl Discov 2(2):121–167 tion. In: Smyth B, Cunningham P (eds) Proceedings of the Euro-
7. Carpenter G, Grossberg S (1987) ART 2: self-organization of sta- pean workshop on case-based reasoning. Springer, Berlin, pp 184–
ble category recognition codes for analog input patterns. Appl Opt 195
26(23):4919–4930 29. Moore AW An introductory tutorial on Kd-trees. citeseer.ist.psu.
8. Corchado J, Lees B, Fyle C, Ress N, Aiken J (1998) Neuro- edu/140157.html
adaptation method for a case-based reasoning system. Comput Inf 30. Moses LE (1986) Comparison of averages from two samples
Syst J 5:15–20 and some related problems. Think and explain with statistics.
9. Breiman L (1996) Bagging predictors. Mach Learn 24(2):123– Addison–Wesley, Reading, Chap. 6
140 31. Orr M (1996) Introduction to radial basis function networks. Tech-
10. Dietterich G (1997) Machine learning research: four current direc- nical report, Centre for Cognitive Science, University of Edin-
tions. AI Mag 18(4):97–136 burgh
11. Main J, Dillom T, Shiu S (2001) A tutorial on case-based reason- 32. Policastro C, Carvalho A, Delbem A (2003) Hybrid approaches
ing. In: Pal S, Dillon T, Yeung D (eds) Soft computing in case-
for cases retrieval and adaptation. In: Günter A, Kruse R, Neu-
based reasoning. Captulo 1. Springer, Berlin
mann B (eds) Proceedings of 26th German conference on artificial
12. Domingos P (1996) Unifying instance-based and rule-based in-
intelligence, Hamburg, Germany. Lecture notes in artificial intel-
duction. Mach Learn 24:141–168
ligence, vol 2821. Springer, Berlin, pp 297–311
13. Duda R, Hart P, Stork D (2001) Pattern classification. Wiley–
33. Quinlan R (1992) Learning with continuous classes. In: 5th Aus-
Interscience, New York
tralian joint conference on artificial intelligence, Hobart, Tasma-
14. Freund Y, Schapire R (1996) Experiments with a new boosting
algorithm. In: Saitta L (ed) 13th international conference on ma- nia. World Scientific, Singapore, pp 343–348
chine learning, Bari, Italy. Kaufmann, San Francisco, pp 148–156 34. Russel R, Norvig P (1995) Artificial intelligence: a modern ap-
15. Hanney K, Keane MT, Smyth B, Cunningham P (1995) What proach. Prentice Hall, Englewood Cliffs
kind of adaptation do CBR systems need? A review of current 35. Smyth B, Cunningham P (1993) Complexity of adaptation in real-
practice. In: Aha DW, Ram A (eds) AAAI 1995 fall sympo- world case-based reasoning systems. In: 6th Irish conference on
sium on adaptation of knowledge for reuse, MIT Campus, Cam- artificial intelligence and cognitive science, Belfast, Ireland
bridge, Massachusetts, EUA. Available at http://www.aic.nrl.navy. 36. Smyth B (1998) Case base maintenance. In: Mira J, Pobil A (eds)
mil/aha/aaai95-fss/papers/hanney.ps.Z 12th international conference on industrial and engineering appli-
16. Hanney K (1996) Learning adaptation rules from cases. Master’s cations of artificial intelligence and expert systems, Cairo, Egypt.
thesis, University College Dublin Springer, Berlin, pp 507–516
17. Haykin S (1999) Neural networks: a comprehensive foundation. 37. Valentini GM, Masulli F (2002) Ensembles of learning machines.
Prentice Hall, New York In: Marinaro M, Tagliaferri R (eds) Neural nets WIRN Vietri-
18. Hilario M (1997) An overview of strategies for neurosymbolic in- 02, Heidelberg, Germany. Lecture notes in computer sciences.
tegration. Connectionist-symbolic integration: from unified to hy- Springer, Berlin
brid approaches. Lawrence Earlbaum Associates, Inc., Chap. 2 38. Vapnik V (1998) Statistical learning theory. Wiley, New York
A hybrid case adaptation approach for case-based reasoning 119

39. Wang Y, Witten I (1997) Induction of model trees for predicting 42. Wilke W, Vollrath I, Althoff K-D, Bergmann R (1996) A frame-
continuous classes. In: Someren M, Widmer G (eds) 9th Euro- work for learning adaptation knowledge based on knowledge light
pean conference on machine learning, Prague, Czech Republic. approaches
Springer, Berlin, pp 128–137 43. Wiratunga N, Craw S, Rowe R (2002) Learning to adapt for case-
40. Watson I (1997) Applying case-based reasoning: techniques for based design. In: Craw S, Preece A (eds) 6th European confer-
enterprise systems. Kaufmann, Los Altos ence on case-based reasoning, Aberdeen, Scotland, UK. Springer,
41. Watson I (1999) CBR is a methodology not a technology. Berlin, pp 421–435
Knowledge-Based Syst 12:303–308