Paper ID: SCT-950

Modeling churn behavior of bank customers using predictive data mining techniques
M Purna Chandar1,2, Arijit Laha1 , P Radha Krishna1
1

Institute for Development and Research in Banking Technology(IDRBT) Castle Hills, Hyderabad-500057 2 Dept. of Computer and Information Sciences, University of Hyderabad, Gachibowli, Hyderabad 500046 mpchandar@mtech.idrbt.ac.in {alaha,prkrishna}@idrbt.ac.in
Abstract Acquiring new customers is a more costlier process than retaining existing customers. Therefore, the management of relationship with customers plays a vital role in improving the overall profitability of a company. There are many segments that come under the Customer Relationship Management (CRM) umbrella such as churn prediction, target marketing, cross/up selling, customer profiling, etc. Churn is defined as the propensity of a customer to cease doing business with a company in a given time period. In this paper, we emphasize on modeling churn behavior of bank customers in Indian scenario. Ideally, many characteristics of customers like demographic details, psychographic (transactional details), product purchase details, customer perception details are vital in modeling the churn behavior of bank customers. But, in real-time, especially in Indian banking scenario, this required data is not captured by the banks. Hence, we have to live with the available data and have to extract the required features from it. In this paper, we give a detailed guideline to convert raw customer data into meaningful data that suits modeling churn behavior. Then, to convert this meaningful data into knowledge, predictive data mining techniques are used. We built models with three decision tree algorithms namely CART, TreeNet and C5.0. CART yielded better classification rate in predicting the churn behavior and C5.0 yielded better classification rate in predicting the non-churn (active) behavior of customers. As predicting churn is more important for a bank, it can be said that CART yielded a better overall classification rate. CART generated seventeen decision rules which can be useful in predicting probable churners. Index Terms Banking, Predictive Data Mining, Churn Prediction, Classification models.

Introduction
High cost of customer acquisition and customer education requires companies to make large upfront investments on customers. However, due to easy access to information and a wide range

Paper ID: SCT-950

of offerings, it is easier than ever before for customers to switch between service providers. This applies to all industry verticals such as banking, telecom, insurance, etc. Therefore, customer churn, which is defined as the propensity of customers to cease doing business with a company in a given time period, has become a significant problem and is one of the prime challenges financial institutions worldwide are learning to face [1]. Studies reveal that customer churn is a costly affair. Identifying the churn before hand and taking necessary steps to retain them (customer retention) would increase the overall profitability of the company. A study conducted in [2] reveal that a bank is able to increase its profits by 85% due to a 5% improvement in the retention rate. Similar findings emerged in [3], who calculated the financial impact of an increase in retention rate of 1%. Another study conducted in [4] says that losing customers not only leads to opportunity costs because of reduced sales, but also to an increased need for attracting new customers, which is five to six times more expensive than customer retention. In the banking industry, identifying probable churn customers has increased in its importance in the recent past. In banking domain, we define a churn customer as one who closes all his/her accounts and stops doing business with the bank. There are many reasons for a customer to close the account(s). For example, a person creates an account for a specific purpose and closes it immediately after the purpose is solved. Or a person is relocated and has to move to another place and hence closes all the accounts. Or a customer may stop transacting with the bank just because of the unavailability of banks ATMs in important places and hence close his/her accounts. The problem here is that, in real world scenario, the bank does not always capture this kind of feedback data. Hence, no further analysis can be done and this type of churning behaviors could not be stopped. This leaves us in a situation where we need to think which kind of churn patterns are possible to identify. Hence prediction of churn becomes a very challenging problem in banking sector. Very often, we need to live with the raw customer data available and extract the churn patterns out of it. So it can be said that the problem of predicting churn is divided into two steps viz., convert the raw data into meaningful data and convert the meaningful data into knowledge. In this paper, we explain the kind of raw data available in realtime banking scenario, provide a guideline to convert raw data into meaningful data and finally convert the meaningful data into knowledge using predictive data mining techniques.

Introducing Predictive Data Mining Data mining is the process of exploration and analysis, by automatic or semi-automatic means, of large quantities of data in order to discover meaningful patterns and rules [5]. It can also be defined as the process of selecting, exploring and modeling large amounts of data to uncover previously unknown data patterns for business advantage [6]. Using data mining, descriptive models and predictive models can be built. Descriptive models are built on the concept of unsupervised learning and predictive models are built on the concept of supervised learning. Here, only predictive data mining models are discussed. In a predictive model, one of the variables (the target variable or response variable) is expressed as a function of the other variables. This permits the value of the response variable to be predicted from given values of the other variables (the explanatory or predictor variables). In the churn prediction problem, the response variable, i.e., the future status of the customers, can take only two values viz. Active or

Paper ID: SCT-950

Churn. Therefore predictive classification techniques are used for churn modeling. In classification, a mapping from a vector of measurements x to a categorical variable Y is learnt. The variable to be predicted is called class variable taking values in a set {c 1, c2 cm} and the observed or measured variables are referred to as attributes or explanatory variables or input variables. In classification, the training data consists of pairs of measurements, each consisting of a vector x(i) with a corresponding target value Y. The goal is to estimate a mapping between Y and x(i) based on the training dataset. This estimated mapping can be called as a model. This model can be used to predict Y given an input vector of measured values x(i) [7]. There are many predictive classification techniques namely nearest neighbor technique, decision tree technique, linear discriminant technique, nave bayes technique, etc. In this paper, we experiment on decision tree technique and hence only this technique is explained here. Among the different approaches for building predictive models using data mining, decision tree based algorithms are commonly used ([8] [12]) to examine the data and to induce a tree and its rules that will be used to make predictions. By observing the number of cases and the class they belong to, in the leaf nodes, the corresponding rules can be generated. A number of different algorithms are used for building decision tree including CHAID (Chi squared automatic interaction detection), CART, Quest and C5.0 [13]. For the purpose of this study, CART, TreeNet and C5.0 are employed in order to build the churn model to help one identify subsegments of the customer base that is likely to churn away, providing a well-identified segment to target with retention programs. The rest of the paper is organized as follows: in section 2 the data preparation methodology is explained, in section 3 the results of various classification techniques are explained, in section 4 the results are discussed and finally we conclude in section 5.

Data Preparation Methodology

As explained in the previous section, data plays a very vital role in building data driven models. For predicting future churn, a very robust model should be in hand and a robust model can only be built if we have a robust dataset in hand. Hence, data preparation is a vital step in churn prediction and it takes almost 60 70 percent of total time. In this section we will explain the kind of data required for robust system, customer data usually available in the real life, guideline to extract useful attributes from the available data. Building a model for churn prediction implicitly means that we are trying to model the behavior of customers before churning out. For this to be successful, the customer transaction activities should be analyzed in a particular period of time. Hence, taking a snapshot of data would never suffice the requirement. On the other hand, considering the transaction activities in a fixed time period would also not suffice the requirement. The reason would be best explained by an example. Say, for example, a model is built using data of 1000 customers of which 700 are active and 300 are known to be churned out and their 3 months activities are analyzed (say, Jan 2005 to March 2005). Here the timeline is fixed and the activities done in this time period of all the 1000 customers are only analyzed. Now, out of 300 churn customers, say 50% of them have churned away in January. This means, the model will not be fully trained with the behavior of churn customers before churning as only one months activity is analyzed. This problem

Paper ID: SCT-950

occurred because of fixing the timeline before hand as shown in figure 1. Many researches in the past were done by preparing the dataset in this fashion. In this paper, we suggest to consider a dynamic timeline, which varies for each customer. This concept would be better explained by continuing the above example. If a customer has churned away in January 2005, from that point of time, the past 3 months activity is considered i.e., transaction activities done in November, December 2004 and January 2005 are considered. And if another customer churns away in February, transaction activity of December 2004 and January, February 2005 should be considered. This concept is shown in figure 2. This way, we adapt a dynamic timeline for each customer and hence avoid the problem of not training the model properly. This concept applies to churn records. For active records, we can consider the behavior in any 3 months period. In our analysis, we consider the behavior of last 3 months before last transaction date of the active customers. The number of months of data to be considered for churn analysis is a business problem. Generally, considering the transaction activities of 3 months would suffice the requirement.

Customer A churned

Customer B churned timeline

Customer A churned

Customer B churned

timeline Time line for customer A Time line for customer B

Fixed timeline for all customers

Fig 1: Fixed timeline for all customers

Fig 2: Dynamic timeline

Next in this section, we explain the real life data we worked with, and filtration steps (data cleansing step) for prepare an efficient dataset. We obtained the customer data from a nationalized Indian bank. The details of the data obtained are shown in Table 1.
TABLE 1 DETAILS OF CUSTOMER DATA Table Name Customer General Ledger Dormant Master Txn Ttype Attributes Custno, Name1, Name2, Address, Status, DoB (Date of Birth), Edn Custno, AcNo, Descr, DOP (Date of a/c opening) Acno, Descr, Dormant Acno, Balance, Dormant flag. Acno, Trntype, Date, Amount Ttype#, Typecode, Descr No. of Records 40,880 1,08,029 18,002 31,022 31,39,020 93

Paper ID: SCT-950

The Customer table contains customer details like customer number, name, address, date of birth, education and status. There are 40,880 customers in total. The general ledger table contains the account numbers, account types, date of opening and description of accounts held by each customer. There are 1,08,029 accounts in all for above specified number of customers. The Dormant table contains all the account numbers that are flagged as inactive (dormant) since a while. The Master table contains account numbers and latest balances of all the accounts. The Txn table contains the previous 5 years transactional details of all the accounts. And finally Ttype table contains description for various transaction types. There are totally 93 transaction types defined. The final dataset prepared from the available data shown in table 1 contains the following attributes: Customer number, Duration (Dur), number of Credit transactions in 3 months (CRTxns), number of Debit transactions in 3 months (DRTxns), Average credit amount in 3 months (AvgCrAmt), Average debit amount in 3 months (AvgDrAmt), total number of other accounts (NumOtherAccs), percentage of accounts closed in 3 months (PercClosedRecently), Status (Status). In the subsequent discussion in this section, we explain how these attributes are populated. The Duration attribute contains the number of months the customer has transacted with the bank. When we say 3 months in an attribute, we refer to the concept of dynamic timeline explained previously. The final dataset contained 1,494 records with 1,173 active customers and 321 churn customers. Extracting this dataset from the raw data available, which is explained above, is not an easy job. Further in this section, we share our experiences in working with the above data and subsequently extracting the data for required set of attributes. There are many discrepancies in this data. First and important of these discrepancies is about the Status attribute in customer table, which is explained below. The status attribute represents whether a customer is active or dormant or churned out (closed all the accounts). Hence status attribute forms the class variable in our dataset. A customer can have one or more accounts with the bank. The problem here was if one of the accounts of a customer is closed, then the status is set as churned out. But in fact the customer is still with the bank because his/her other accounts are still active. Hence it was realized that this status attribute couldnt be used in preparing the training dataset. But status of each account is very much required as this forms the class variable of the dataset. The Descr attribute in General Ledger table helped to find the status of each account. The Descr attribute contained the description of the account type and also whether it is closed or still active. Hence the status attribute in the dataset is populated using Descr attribute of the General Ledger table. Second problem with the data was some of the fields contained missing values. Fields like Edn, DoB in Customer table, DOP in General Ledger table were sparsely populated. Due to this reason, demographic attributes like age (can be derived from DoB), education, gender could not be taken in the attribute list of final dataset. The data contained many types of accounts like savings accounts, current accounts, cash credit accounts, loan accounts, etc. In this study, the behavior of savings accounts customers is analyzed. Out of 1,08,029 total accounts, number of savings accounts are 22,165. In this, 6,643 accounts are found to be dormant from Dormant table. In this analysis, we are not interested in modeling the behavior of dormant customers. Hence these accounts are removed from our target customer base. There are some accounts whose Duration (number of months transacted with the

Paper ID: SCT-950

bank) is very less. This gave an impression that these accounts are opened for a specific purpose and closed immediately. Considering these accounts behavior may lead to a poor dataset and subsequently poor predicting model. Hence all the records whose duration is less than 6 months are removed. There is another set of accounts where duration is quite big but not much transaction activity went in these accounts. This gave an impression that these customers have opened savings account and have very rarely transacted through them. This kind of data may also result in poor modeling. Hence all accounts with transaction count less than 50 are removed from target customer base. After applying the above filtering steps, our target customer base size reduced to 1,494 accounts with 1,173 active records and 321 churn records. The transactional behavior of these 1,494 customers is analyzed by extracting the right kind of data from the available raw data. As said earlier, in order to model the behavior of both churn and active customers, we have to train the model with their recent behavior. The recent behavior is extracted from the Txn table with contains the transactional details of all the accounts. Each record in the Txn table contains the Trntype field which says the type of the transaction (credit voucher, inward cheque clearing, cash deposit are some the values the Trntype field can take). There are 93 distinct transaction types and each can be identified as either a credit type of transaction or a debit type of transaction. We segregated the transactions done by customers into credit transaction or a debit transaction and eventually calculated the number of credit transactions (CRTxns) and number of debit transactions (DRTxns) for all the 1,494 customers. The average amount of money transacted by customers in the considered time period may also help in training the model better. Hence, two more attributes namely the average amount involved in credit transactions (AvgCrAmt) and the average amount involved in debit transactions (AvgDrAmt) were calculated for 1,494 customers. It was said earlier that we have considered the transaction types of only savings accounts of the customers. But these customers may also have others accounts with the bank. The number of other accounts and the percentage of accounts closed in the considered time period may train the model with another important pattern. Hence two more attributes namely number of other accounts (NumOtherAccs), percentage of other accounts closed in considered time period of 3 months (PercClosedRecently) were calculated for all the 1,494 customers.

Building Data Mining Models And Experimentation Results

In the previous section, we have explained the data preparation steps in detail. Having a dataset readily in hand, the next step is to build a well trained predictive model using the dataset. Data mining techniques are used to build data driven models and subsequently use them for future predictions. As explained previously, the problem of predicting churn is purely a classification problem and supervised learning data mining techniques are used to solve classification problems. We used three classification tree algorithms namely CART, C5.0 and TreeNet for building the models. The dataset is split into training dataset and testing dataset in 80 20 ratio in building the three classification trees.

Paper ID: SCT-950

Classification tree model using CART Classification tree is built using CART on the training dataset using the following specifications: CART uses no stopping rule that could be relied on to discover optimal tree. So the tree is over grown and then pruned back which ensures that important patterns are not overlooked by stopping too soon. CART does binary splitting that are more sparing with data and detect more patterns before too few data are left for learning. It produces rules for the target variable as a function of other fields in the dataset that are previously identified as explanatory variables. Gini concentration coefficient is used to summarize power curves of prediction. Customer Duration, CRTxns, DRTxns, AvgCrAmt, AvgDrAmt, PercClosedRecently are used as explanatory variables and Status is taken as target variable. 80% of the dataset i.e., 1,192 samples containing 929 active customer records and 263 churned customer records, are taken in training dataset. The remaining 20% of the dataset i.e., 299 samples containing 244 active customer records and 58 churned customer records, are taken in testing dataset. The confusion matrix and classification success rate of training dataset and testing dataset are shown in table 2 and table 3 respectively.
TABLE 2 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TRAINING DATA True Class Active Churn Total # samples 929 263 Predicted Active 800 13 Predicted Churn 129 250 Success % 86.11 95.05 True Class Active Churn TABLE 3 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TEST DATASET Total # samples 244 58 Predicted Active 210 5 Predicted Churn 34 53 Success % 86.06 91.37

The success rate of active customers is relatively less. This is because, some of the customers, although their status is marked as active, exhibited churn characteristics. It is this segment of customers on which bank has to concentrate upon and apply churn prevention methodologies. There are 17 leaf nodes in the tree generated using CART and hence 17 decision rules can be drawn from it. Table 4 shows the 17 decision rules.
TABLE 4 DECISION RULES GENERATED BY CART Rule # 1 2 3 4 5 6 Rule AvgDrAmt <= 608 and AvgCrAmt <= 37.5 AvgDrAmt <= 608 and AvgCrAmt > 37.5 and AvgCrAmt <= 1655.5 and Duration > 18 AvgCrAmt <= 1655.5 and AvgDrAmt > 608 AvgCrAmt > 1655.5 and Duration <= 23.5 and AvgDrAmt <= 1300.5 AvgCrAmt > 1655.5 and Duration <= 23.5 and AvgDrAmt > 1300.5 and PercClosedRecently > 0.0416667 AvgCrAmt > 1655.5 and Duration > 23.5 and Duration <= 27.5 Predicted Class Churn Active Churn Active Churn Active # Cases 93 61 102 20 26 615

Paper ID: SCT-950

7 8 9 10 11 12 Duration > 27.5 and Duration <= 68.5 and AvgDrAmt <= 3421.5 and AvgCrAmt > 1655.5 and AvgCrAmt <= 3674 Duration > 27.5 and Duration <= 68.5 and AvgDrAmt <= 3421.5 and AvgCrAmt > 3674 AvgCrAmt > 1655.5 and Duration > 27.5 and Duration <= 68.5 and AvgDrAmt > 3421.5 AvgCrAmt > 1655.5 and Duration > 68.5 AvgDrAmt > 1300.5 and PercClosedRecently<= 0.04 and AvgCrAmt > 1655.5 and AvgCrAmt <= 17894.5 and Duration <= 17.5 PercClosedRecently <= 0.04 and Duration > 17.5 and Duration <= 23.5 and AvgDrAmt > 1300.5 and AvgDrAmt <= 7449 and AvgCrAmt > 3527 and AvgCrAmt <= 17894.5 PercClosedRecently <= 0.04 and Duration > 17.5 and Duration <= 23.5 and AvgDrAmt > 1300.5 and AvgDrAmt <= 7449 and AvgCrAmt > 1655.5 and AvgCrAmt <= 3527 AvgDrAmt <= 608 and AvgCrAmt > 37.5 and AvgCrAmt <= 1655.5 and Duration <= 18 PercClosedRecently <= 0.04 and AvgCrAmt > 1655.5 and AvgCrAmt <= 17894.5 and Duration > 17.5 and Duration <= 23.5 and AvgDrAmt > 7449 Duration <= 23.5 and PercClosedRecently <= 0.04 and AvgCrAmt > 17894.5 and AvgDrAmt > 1300.5 and AvgDrAmt <= 14238.5 Duration <= 23.5 and PercClosedRecently <= 0.04 and AvgCrAmt > 17894.5 and AvgDrAmt > 14238.5 Churn Active Churn Active Churn Active 16 18 38 43 70 32

13

Churn

10

14 15 16 17

Churn Churn Active Churn

4 10 10 10

Out of the 17 rules generated by CART, 12 rules have sufficient number of cases and these rules can be used as rules of thumb by the manager for predicting probable future churn customers.

Classification Tree Model using TreeNet TreeNet uses stochastic gradient boosting algorithm as a technique for improving the accuracy of a predictive function by applying the function repeatedly in a series and combining the output of each function with weighting so that the total error of the prediction is minimized. In many cases, the predictive accuracy of such a series greatly exceeds the accuracy of the base function used alone. The stochastic gradient boosting algorithm used by TreeNet is optimized for improving the accuracy of models built on decision trees. Number of trees formed by TreeNet was set to 200 and the regression loss criterion used was Huber-M. The confusion matrix and classification success rate of training dataset and testing dataset are shown in table 5 and table 6 respectively.

Paper ID: SCT-950

TABLE 5 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TRAINING DATA True Class Active Churn Total # samples 929 263 Predicted Active 809 24 Predicted Churn 120 239 Success % 87.08 90.87 TABLE 6 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TEST DATA True Total # Predicted Predicted Success Class samples Active Churn % Active Churn 244 58 218 6 26 52 89.34 89.65

3.1.

Classification tree Model using C5.0

C5.0 is another classification algorithm used to generate decision trees. Unlike CART, this algorithm produces trees with variable branches per node. Customer Duration, CRTxns, DRTxns, AvgCrAmt, AvgDrAmt, PercClosedRecently are used as explanatory variables and Status is taken as target variable. 80% of the dataset i.e., 1,192 samples containing 929 active customer records and 263 churned customer records, are taken in training dataset. The remaining 20% of the dataset i.e., 299 samples containing 244 active customer records and 58 churned customer records, are taken in testing dataset. The confusion matrix and classification success rate of training dataset and testing dataset are shown in table 7 and table 8 respectively.
TABLE 7 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TRAINING DATA True Total # Predicted Predicted Success Class samples Active Churn % Active 929 883 46 95.04 Churn 263 61 182 69.2 TABLE 8 CONFUSION MATRIX AND PREDICTION SUCCESS RATE FOR TEST DATA True Total # Predicted Predicted Success Class samples Active Churn % Active 244 234 10 95.9 Churn 58 18 40 68.9

Discussion Of Results
In this study, we have experimented with 3 classification techniques namely CART, TreeNet and C5.0. The prediction success rate of Churn class by CART and TreeNet are quite high but C5.0 had shown poor results in predicting churn customers. However, the prediction success rate of Active class by C5.0 is more than the other two techniques. But for reaping significant benefits, the model should be able to predict the churn behavior better. Hence, a model with higher prediction success rate of Churn class has to be chosen for reaping higher benefits. In all the decision tree models, all the explanatory attributes were found to be influencing the target variable, i.e., status of the customer.

Conclusions

Paper ID: SCT-950

Transforming raw customer data into usable information is the goal of many data mining projects. But failure to turn this information into customer satisfaction and increased profits is the key to why many such projects often fall short of expectations. Thus, it is imperative that the customers, indicated by the churn model, to become churn should be focused upon. If the churn prevention programme is effective, the bank can look forward to reaping significant benefits from its efforts. A company that can retain as few as 5% of its current customers can raise its profits by as much as 25%. In this paper, we have given a detailed guideline of converting raw customer data of a public sector bank into useful data and then convert this useful data info useful information using data mining techniques. We have explained the concept of dynamic timeline that should be considered while converting raw data into useful data. We have extracted the data for chosen attributes from raw customer data for a chosen set of 1,494 customers. In these 1,494 samples, 1,173 customers have the status as active and 321 customers have the status as churn. We used CART, TreeNet and C5.0 to identify key customer characteristics to predict churn. While CART yielded 95.05% classification rate on training data and 91.37% on test data, TreeNet yielded 90.87% classification rate on training data and 89.65% on test data, C5.0 yielded 69.2% classification rate on training data and 68.9% on test data. The study thus predicts the future churn for banking customers and can then be extended, thereby helping formulate intervention strategies based on churn prediction to reduce the lost revenue by increasing customer retention. It is expected that, with a better understanding of these characteristics, bank managers can develop a customized approach to customer retention activities within the context of their Customer Relationship Management efforts.

References
[1]. J.Dyche: The CRM handbook: a business guide to customer relationship man`agement. Reading, MA: Addison-Wesley, 2001. [2]. Reichheld,F. F., & Sasser, W.E., Jr., (1990) Zero defections: quality comes to service, Harvard Business Review, 68(5), 105 - 111. [3]. Van den Poel, D., & Lariviere, B., Customer attrition analysis for financial services using proportional hazard models, European Journal of Operational Research, 157 (2004), 196 217. [4]. Athanassopoulos, A.D., 2000, Customer satisfaction cues to support market segmentation and explain switching behavior, Journal of Business Research, 47 (3), 191 207. [5]. Berry, M.J.A. and Linoff, G. (2000), Mastering Data Mining: The Art and Science of Customer Relationship Management, Wiley Computer Publishing, New York, NY. [6]. SAS Institute (2000), ``Best practice in churn prediction, A SAS Institute White Paper. [7]. David Hand, Heikki Mannila, Padhraic Smyth. (2001). Principles of Data Mining.

Paper ID: SCT-950

[8]. J. Gehrke, V. Ganti, R. Ramakrishnan, and W.-Y. Loh, BOAT Optimistic decision tree construction, in Proc. ACM SIGMOD Int. Conf.Management of Data, Philadelphia, PA, 1999, pp. 169180. [9]. J. Gehrke, R. Ramakrishnan, and V. Ganti, RainForest A framework for fast decision tree construction of large datasets, in Proc. 24th Int.Conf. Very Large Data Bases, New York, 1998, pp. 416427. [10]. J. Shafer, R. Agrawal, and M. Mehta, SPRINT: A scalable parallel classifier for data mining, in Proc. 22nd Int. Conf. Very Large Data Bases, Mumbai (Bombay), India, 1996, pp. 544555. [11]. J. R. Quinlan, Decision trees as probabilistic classifiers, in Proc. 4th Int. Workshop Machine Learning, Irvine, CA, 1987, pp. 3137. [12]. R. Rastogi and K. Shim, PUBLIC: A decision tree classifier that integrates building and pruning, in Proc. 24th Int. Conf. Very Large Data Bases, NewYork, 1998, pp. 404415. Olivia Parr Rud, Data Mining Cook Book, John Wiley and Sons Nov, 2000.

[13].