2021 2nd International Conference on Artificial Intelligence and Data Sciences (AiDAS)
Predictive Model using SVM to Improve the
Effectiveness of Direct Marketing
1st Noviyanti Tri Maretta Sagala
Statistics Department, School of Computer Science
Bina Nusantara University
Jakarta, Indonesia 11480
noviyanti.sagala@binus.edu
Abstract—Direct marketing through phone calls is one of the
most general marketing campaigns. Identifying profitable or
risky customers are needed to improve the success of bank
2021 2nd International Conference on Artificial Intelligence and Data Sciences (AiDAS) | 978-1-6654-1726-6/21/$31.00 ©2021 IEEE | DOI: 10.1109/AIDAS53897.2021.9574440
telemarketing. In this study, a comprehensive data analysis is
proposed before developing a predictive model on Portuguese
retail bank. The process starts by understanding business and
data used, followed by data preparation, then apply SVM
algorithm to build predictive models. During business and data
understanding, how the business works is analyzed, the problem
of the bank and gather the data needed for modelling. Later,
univariate, bivariate, and multivariate analysis are conducted to
have a general idea about the variables. Data preparation is
performed to ensure the predictive models built comes from a
good quality data set. Two schemas with SVM are applied on
the data set. The performance of models is assessed by the
specificity, sensitivity, and accuracy rate. From the
experimental results, SVM with hyper tune parameter achieves
specificity at 82.5, sensitivity at 72.4, and accuracy at 76.48%.
Keywords— bank telemarketing, data analysis, direct
marketing, GridSearch, User-defined function, SVM
I. INTRODUCTION
The use of data-driven decision-making has become a
critical component of critical responses to the challenging
business environment. To maximize profits, maintain a long-
term relationship with customers, and sustain a competitive
advantage, businesses are required to identify profitable or
risky customers.
Telemarketing is one of many formats of direct
marketing strategy where a salesperson makes a phone call to
influence customers to accept to a long-term deposit [1].
Direct marketing is a form of promotional strategy that aims
to elicit a particular response from a target group of customers
(such as a request, a visit to a location or platform, or a request
for information) in reply to a salesperson's communication
action.
The most effective telemarketing focuses on prospect
data quality, intending to estimate the expected customers
who are more likely to use the service using data mining
techniques. Many banks have used a predictive technique
based on data mining to categorize customers before going to
offer special services to better understand their behavior
[2][3].
The study proposes a comprehensive data analysis
framework and investigates the effectiveness of the two
schemas of SVM predictive models in identifying a group of
clients who have a high likelihood of subscribing to a long-
term deposit. The data used for this study is a dataset publicly
available for research and was collected from a Portuguese
retail bank. The performance of the predictive model is
measured by sensitivity, specificity, and accuracy.
978-1-6654-1726-6/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
2nd Syarifah Diana Permai
Statistics Department, School of Computer Science
Bina Nusantara University
Jakarta, Indonesia 11480
syarifah.permai@binus.ac.id
The remainders of the paper are organized as follows:
section I describes the background and related works to the
study followed by section II which specifies the proposed
methodology for this work. The experimental results and
discussion are reported in section III; Lastly, the summarize
work and suggested future works are in section IV.
II. RELATED WORKS
The most significant task in data mining is prediction or
classification, which is frequently used to categorize a bunch
of data. The quality and quantity of input features are two
crucial elements that influence the success of prediction task.
The information of bank customers is known to include a lot
of attributes, which reduces the performance of predictive
model. Furthermore, those attributes were unsatisfactory to
build predictive models.
An intelligent system for predicting bank telemarketing
success was developed using Logistic Regression [4]. The
predictive model on the success of bank telemarketing was
built using ANN [5]. Cost-sensitive analysis was utilized to
reduce the impact of extremely unbalanced data on bank
campaign predictive model. The methodology aims to
provide banks telemarketing decision makers with additional
information about their marketing operations [6]. The similar
dataset was used to build predictive model of customer
behavior. From the experimental result the accuracy of
Neural Networks classification model is comparatively better
than Naïve Bayes model [7].
SVM is a unique classification model above all others
because of its capacity to generalize on unknown data and
accomplish tasks such as classification and regression.
Furthermore, the approach distinguishes between two classes
for which it is typically appropriate. The SVM is one of the
most well-known classification algorithms [8].
The performance time of a SVM model for
classification is usually longer and is proportional to the
number of support vectors, which might be troublesome for
some ongoing applications. SVM generates the dot product
for the supplied test point with each of the support vectors
either in the feature space or in the space of input after
conversion through a function called kernel for the goal of
classifying the data point. As a result, the execution time
increases as the number of support vectors increases [9].
III. METHODOLOGY
A. Dataset Description
The dataset utilized in this study are real data downloaded
from UCI Machine Learning repository. It was accumulated
from a Portuguese retail bank more than five years, from May
2008 to June 2013 [10]. The dataset was gathered by
marketing campaign of a Portuguese financial organization
based on phone calls. The dataset contains 41188 telephone
contacts made of 21 features with a label attribute ‘yes’ and
‘no’ as demonstrated in Table 1.
TABLE I. DATASET DESCRIPTION
Type of
No Variable Description
variable Fig. 1. The proposed methodology
1 Age Age of Client Numeric
1) Business Understanding: The goal of business
2 job Type of job (admin.’,’blue- Categorical understanding phase is to define the business goals that the
collar’,’entrepreneur’,’housemaid’,’
management’,’retired’,’self- data science techniques can target. Basically, the need of
employed’,’services’,’student’,’tech knowing where the information comes from, what decisions
nician’,’unemployed’,’unknown) are being made, what exactly is the problem with the current
3 marital Marital status Categorical
(divorced’,’married’,’single’,’unkno decisioning process, find the relevant data that needs to solve
wn’; note: ‘divorced’ means the questions that define the objectives of the project, and
divorced or widowed) define the success metrics.
4 education basic.4y’,’basic.6y’,’basic.9y’,’high. Categorical
school’,’illiterate’,’professional.cour
2) Data Understanding: The aim of data understanding
se’,’university.degree’,’unknown phase is to create a clean, high-quality data collection with a
5 default Has credit in default? Categorical well-understood link to the target variables. At this phase,
6 housing Has housing loan? Categorical the need to develop a sound understanding of the information
gathered from the previous phase. Checking on the type of
7 loan Has personal loan? Categorical
data to learn more about the attributes and their names is
8 Contact Contact communication type Categorical needed.
(cellular, telephone) 3) Data preparation: Data Preparation is the process of
9 month Last contact month of year (jan to Categorical
dec) exploring, cleaning and transforming raw data prior to
10 Day_of_ Last contact day of the week (mon to Categorical processing and analysis. Data may come from several
week fri) sources, large in number and have a wide range of
11 duration Last contact duration, in seconds Numeric
characteristics. Commonly, data from the real world is
12 campaign Number of contacts performed numeric chaotic, unreliable, and contradictory. It means the raw data
during this campaign and for this is prone to corruption, lacking values or attributes, outliers,
client (includes last contact)
13 pdays Number of days that passed by after Numeric or values that are inconsistent. The data preparation stage
the client was last contacted from addresses these questions, ensuring that the dataset used in
14 previous Number of contacts performed numeric the modeling stage is suitable and of high quality. It is a
before this campaign and for this
client
crucial task to produce valid and reliable models that have
15 poutcome Outcome of the previous marketing Categorical high accuracy and efficiency. By having excellent quality
campaign data, it leads to more timely, efficient, and high-quality
16 Emp.var.r Employment variation rate- quarterly numeric business decisions. In this phase, univariate, bivariate, and
ate indicator
17 cons.pric consumer price index — monthly numeric multivariate analysis by utilizing data visualization and
e.idx indicator statistical techniques are performed and followed by several
18 Cons.con Consumer confidence index numeric steps as follows:
f.idx
a) Renaming variables. The intention of renaming the
19 euribor3 euribor 3 months rate — daily numeric
m indicator features is to have good variable names which are
20 nr.emplo number of employees — quarterly numeric ease of recall and understandable. Total of 9 variables
yed indicator are renamed.
21 Label Has the client subscribed a term Categorical
deposit? b) Getting rid of duplicated rows. Duplication simply
means that the database includes repeated data. This
B. The Proposed Methodology may be attributed to data entry errors or data
Basically, the proposed methodology contains (a) processing processes. Deduplication may lead to
Business understanding, (b) Data understanding, (c) erroneous assumptions by letting you assume that such
Data Preparation, (d) Modelling & Evaluation. Fig. 1 findings are more widespread than they are. The
summarizes the main steps of the experiments. technique used is find and remove exact duplicated.
Exact duplicate means more than one record that is
exactly the same.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
c) Mode imputation. It operates by replacing missing
values of categorical variables by the model of non-
missing cases of that variable [11]. Values could be
absent for a variety of purposes, all of which are
unique to the issue domain, such as corrupt
calculations or data inaccessibility. There are several
ways to deal with missing values. One of them is data
imputation. Using statistical techniques to
approximate a value for a column from the values that
are available, then replacing all missing values in the
column with the estimated statistic, is a basic and
common approach to data imputation. Statistical
techniques often prove very effective technique due to
fast calculation. The 7 categorical features that contain
missing values are job, marital, education, default,
housing, loan, and outcome last campaign.
d) Removing unimportant variables: There are two
conditions to state whether variables are insignificant.
First, the variable is lack of group diversity and second
is having high correlation with other variables. Those
condition are fulfilled by feature default.
e) Recoding values of attributes: In some cases, some
attributes may consist of unbalanced number of
observations for each group/category. Therefore, those
attributes may need to be converted into other form to
have better meaning. Initially, two or three groups
with the justification of the distribution of value in
original groups are created. The first group that created
is for values with the highest number of records, then
second or third group is created for remaining values
based on analysis results. Features previous, last
contact, and age are recoded in this study.
f) Removing outliers. An outlier is a data point in a
dataset that stands out from the rest of the data or
observations. Outliers from feature duration and
campaign are removed.
g) Encoding: The primary reason the need of converting
categorical columns to numerical columns or also
known as encoding is to ease a machine learning
algorithm understands it. For this work purposes, One-
Hot encoding is implemented.
h) Feature Scaling: It is a technique to standardize the
independent features present in the data in a fixed
range. It is used to accommodate highly variable
magnitudes, values, or units during data pre-
processing. If function scaling is unexecuted, a
machine learning algorithm would consider larger
values to be bigger and smaller values to be lower,
regardless of the unit of measurement.
Standardization or commonly known as Z-score is
proposed in this work. The method works
by clustering the values around the mean with a unit
standard deviation [12]. As a consequence, the mean
of attribute becomes zero, and the resulting
distribution has a unit standard deviation. The formula
of standardization is as follows:
 
 =

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
where  is the mean of the feature values and  is the
standard deviation of the feature values. Note that in this case,
the values are not restricted to a particular range.
i) Dimensionality Reduction. It refers to techniques that
reduce the number of input variables in a
dataset. Principal Component Analysis (PCA) is
implemented in this study.
i) Data balancing. A classification problem in which the
classes (i.e., the proportion of defective and clean
modules) are not evenly distributed is known as class
imbalance. For this study purposes, ROSE library is
used.
4) Modelling & Validation: In this phase, machine
learning is used to find patterns and apply rules to the data via
algorithms and then tested on sample data. The Support
Vector Machine(SVM) algorithm proposed in this study is
implemented through the R language. This classification
model focuses on predicting the target through telemarketing
to sell the term deposits. In the matter of model evaluation,
the performance of SVM based on specificity, sensitivity, and
accuracy is measured.
C. ROSE(Random Over-Sampling Examples)
One of popularly used class rebalancing techniques is
ROSE (Random Over-Sampling Examples). In the field of
software engineering, recent literature reveals that bootstrap
resampling techniques yield more precise and consistent
estimates [13]. Recently, a study showed that a smoothed
bootstrap resampling technique (ROSE) outperforms other
techniques in a non-software engineering domain [14]. Thus,
ROSE technique is used in this work [15]. The ROSE
technique draws artificial samples from the function space
neighborhood around the minority class using a smoothed-
bootstrapping method. By generating an augmented sample
of the data (especially from the rare class), ROSE combines
oversampling and under sampling. There are three steps to
the ROSE technique:
Step 1: Using a bootstrap resampling method, resampling
the data of the majority class to eliminate modules from the
majority class with a faulty ratio of 50%.
(underrepresentation)
Step 2: Using a bootstrap resampling technique to
replicate modules of the minority class to a faulty ratio of
50%, resampling the data of the minority class.
(oversampling)
Step 3: Creating new synthetic data in its neighborhood,
with the shape of the neighborhood defined by the ROSE R
package.
These three steps are repeated with each training sample
to create a new synthetic training sample that is about the
same size as the initial dataset and has the same number of
modules for both groups (i.e., a defective ratio of nearly
50 percent).
D. One-Hot Encoding
One-hot encoding is the process of creating dummy
variables. It is the most extensively used coding system. Each
level of the category variable is compared to a predefined
reference level. A single variable with n observations and d
(1) different values is transformed into d binary variables with n
observations each using a single hot encoding. Each
 observation shows whether the dichotomous binary variable It is important to observe Fig. 2 that call duration during
is present (1) or absent (0) [16]. previous campaign was between 0 and 35 minutes. Most of
the clients was called below 5 seconds and only a few was
E. PCA
called longer than that.
One of the most used technique for data-dimensionality
reduction is PCA. It is commonly referred to as a preprocess
for pattern identification and prediction using machine
learning algorithms (MLA). The approach allows you to
minimize the number of variables in a multivariate dataset
while maintaining as much variance as feasible. The basic
process of PCA is given below [17]:
• Normalize the input data set such that each attribute has
the same range of values i.e.: 0-1.
• For the normalized data, compute k orthogonal vectors
that are perpendicular to each other. The k-orthogonal
vectors will be the principal components. Fig. 3. Total clients over education level
• Sort the principal components according to their
decreasing variance. The first axis will represent the According to Fig. 3, clients were university graduates
component with the highest variance, and the second axis with about 12,000 clients. Followed by high school graduates
will be the component with the second greatest variance, with almost 10,000 clients. In short, the most customers were
and so on. from secondary and tertiary educational level.
The results of bivariate analysis for day and month
• Remove the weaker components in terms of variance to
contact of the year as well as age and the status of
obtain a reduced representation of the original data.
When producing principal components, the PCA procedure subscription are shown in Fig. 4 and Fig. 5.
employs metrics such as the covariance matrix, Eigen values,
and Eigen vectors.
F. Evaluation Performance Measurement
For the model evaluation measurement, sensitivity,
specificity, and accuracy are used. The number of items
categorized as true positive in proportion to all items is called
sensitivity (true positive rate) or recall [18]. Sensitivity is a
prominent tool for assessing performance [19]. While
specificity (true negative rate) is a measure of the proportion
of the classifier predicted a Negative class out of all the times
the class was is Negative. It is handy for showing what Fig. 4. Bivariate analysis of last contact month and day of the year
proportion of positive instances were discovered. As a result, It is interesting to note from Fig. 4 that no contact has
a high recall score suggests that only a small number of been made during January and February. In addition, the
positive samples were discarded [20]. Accuracy is the marketing staff gave a call during weekdays. It is clearly
number of valid predictions made by the model out of the displayed those clients had lots of calls on May especially on
total amount of observations [21]. Wednesday and Friday. On the contrary to December, there
are enough observations to conclude this is not pure luck, so
IV. RESULT AND DISCUSSION
this feature will probably be very important in models.
In this section, results of experiment are introduced in
detail. By running code in the R language software, univariate From Fig. 5, the age range of clients is extensive, from
analysis for attributes duration and education level are 18 to 95 years old. However, most customers called is in the
conducted as shown in Fig. 2 and Fig. 3. age of 30s and 40s (33 to 48 years old fall within the 25th to
75th percentiles). The distribution of customer age is normal
with a small standard deviation.

Fig. 2. Univariate analysis for duration Fig. 5. Bivariate analysis between education level and status of customers

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
 The multivariate analysis is carried out. The outcome of The result of removing outliers is displayed in Fig.7 and
the analysis is in the form of correlation plot as visualized in Fig. 8.
Fig. 6.

Fig. 6. Correlation plot among numerical features

The cutoff of 0.5 is set. Consequently, features with

correlation coefficient values higher than 0.5 will be removed
from the data set. The correlation plot shows the correlation
value between numEmp-EmpVaRate and numEmp-
threeMonthrate are 0.97 and 0.95, respectively. Followed by
EmpVaRate-threeMonthRate and EmpVaRate-priceIdx are Fig. 7. Feature campaign with & without outliers
0.91 and 0.78, in a sequential order. Correlation values of
numEmp-priceIdx at 0.69 as well as threeMonthRate-
priceIdx at 0.52. In total, there are four features that are
eliminated from the data set at the end of this step namely
numpEmp, EmpVaRate, 3MonthRate, and PriceIdx.
After completing data exploration phase, several steps
during data preparation stage are performed. Initially, several
variables are renamed for simplicity purpose. Next, removing
12 duplicated rows and operating a mode imputation
technique to handle missing value from categorical variables.
The mode of categorical variables that contain missing i.e.,
Job, Marital, Education, Default, Housing, Loan, Outcomes
Last Campaign variables respectively are admin, married,
university, no, yes, no, and success.
Later, variable ‘default’ is certainly not usable since the
variable has solely 3 individuals replied “yes” to the question
“Do you have credit in default?”. People either answered
“no” or do not even reply, which gives zero information in
our case. This variable is removed from the data set. It is
important to observe that there are 3 numeric variables that
need to be recoded. The reasons are the variables are
meaningless in numeric formats and unbalanced number of
observations. If they are remained unchanged, the value with
high number of observations will bring bias to the predictive
model. As the result, feature previous and last contact were Fig. 8. Feature duration with & without outliers
converted into 0 and 1. Clients who are unsubscribed from
the previous campaign remained set unsubscribed while the One-hot encoding was executed afterwards and
remaining observations were set 1 or subscribed. In addition, produced 43 features that is larger in quantity than the
clients who was not previously contacted or ‘999’ was original one. From the observation, feature ConfIdx
changed to 0 (unsubscribed) while the remaining clients was (Consumer price index) had wider range of values than other
modified to 1(subscribed). Thereby, both classes have almost numeric features. The result of feature scaling technique was
equal number of observations. On the contrary to the previous the reduced range into between -2.2 and 2.9.
actions for recoding step, converting variable age into 3 At the end of data preparation stage, ROSE library is
categories namely level 1 (customer aged less than 30 years used. The parameters method = ‘both’ and seed = 1 to
old), level 2(customer aged between 30 and 60 years old), and generate data synthetically are set. When method = “both” is
the remaining ‘age’ into level 3 are proposed.

Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
selected, both the minority class is oversampled with
replacement and the majority class is under sampled without
replacement.
In modeling phase, two schemas by using SVM
algorithm are proposed. The schemas are SVM with user-
defined function and SVM with hyper tune parameters using
GridSearch technique. The 80% of cleaned data set are used
for model building and the remaining set for model testing.
To use SVM in R, package ‘e1071’ was used. To offer further
insight into the model evaluation measurement, the result is
shown in table II.
Table II. Model evaluation using SVM algorithm.
Schema Cost Gamma Specificity Sensitivity Accuracy
User- 1 1 0.833 0.716 0.7616
defined
GridSearch 0.5 16 0.825 0.724 0.7648
Overall, in terms of sensitivity and accuracy, model with
GridSearch schema is outperformed than user-defines
schema. Model with hyper tune parameters that is 72.4%
sensitive will identify 72.4% clients who are subscribed while
71.6% clients who are subscribed are identified by model
with user-defined schema.
Furthermore, a test conducted using user-defined schema
correctly reports 83.3% of clients who are unsubscribed as
true negatives, which is slightly better than model with
GridSearch at 82.5% specificity. Regarding the accuracy of
the models, GridSearch schema is somewhat higher than
user-defined, at 76.48% and 76.16% respectively.
V. CONCLUSION
In this study, a comprehensive data analysis for predictive
models is proposed. Initially, understanding business and
data, then conduct data exploration and modelling and ends
with model evaluation. Followed by developing 2 schema of
predictive modelling and judge the best schema according to
the sensitivity, specificity, and accuracy of the model. The
prediction model built using hyper tune parameter SVM in
this research achieved reasonable performance. The
specificity at 82.5, sensitivity at 72.4, and accuracy at
76.48%. It may be used by banks and other relevant
businesses to identify a group of clients who have a high
likelihood of subscribing to a long-term deposit. However,
other types of classification model which have been included
or not included in this study should be explored further.
Therefore, the experiment still needs improvement, which
means the author can work with a bigger data size.
Specifically, more personal financial records such as income,
credit card bills, payments.
REFERENCES
[1] V. Abraham and J. Joseph, “An empirical study on direct marketing as
the most effective form of marketing in the digitalized marketing
environment,”Intl. J. of Research Sci. & Management, vol. 6, no. 1, pp.
18–24, Jan. 2019.
Authorized licensed use limited to: National Cheng Kung Univ.. Downloaded on December 06,2021 at 05:55:09 UTC from IEEE Xplore. Restrictions apply.
[2] D. Farid, H. Sadeghi, E. Hajigol, and N. Z. Parrooy, “Classification of
bank customers by data mining: a case study of Mellat Bank branches
in Shiraz,” Intl. J. of Management, Accounting and Economics, 3(8),
534-543.
[3] M. M. Hassan and T. M, “Predictive Risk Categorization of Retail
Bank Loans Using Data Mining Techniques,” Intl. J. of Sci. Research
in Comp. Sci., Eng. and Inf. Tech., vol. 4, no. 1, pp. 302–306, Apr.
2018.
[4] Y. Jiang. Using Logistic Regression model to predict the success of
bank telemarketing,” Intl. J. on Data Science and Tech, vol. 4, no. 1,
pp. 35-41, 2018
[5] M. Selma, “Predicting the success of bank telemarketing using
Artificial Neural Network,” Intl. J. of Eco. And Management Eng, vol.
14, no. 1, pp. 1–4, Jan. 2020.
[6] N. Ghatasheh, H. Faris, I. AlTaharwa, Y. Harb, and A. Harb, “Business
analytics in telemarketing: Cost-Sensitive analysis of bank campaigns
using Artificial Neural Networks,” Appl. Sci., vol. 10, no. 7, pp. 2581,
2020.
[7] T. F. Bahari and M. S. Elayidom, “An efficient crm-data mining
framework for the prediction of customer behaviour,” Procedia Comp.
Sci., vol. 46, pp. 725–731, 2015.
[8] N. T. Sagala, “A comparative study of data mining methods to diagnose
cervical cancer,” J. of Physics: Conf. Series, vol. 1255, pp. 1-6, 2019.
[9] N. Sagala, & J. H. Wang, “A comparative study for classification on
different domain,” Proceedings of the 2018 10th Intl. Conf. on Machine
Learning and Computing, 2018.
[10] S. Moro, P. Cortez, and P. Rita, “A data-driven approach to predict the
success of bank telemarketing,” Decision Supp. Sys., vol. 62, pp. 22–
31, 2014.
[11] S.I. Khan, Hoque, A.S.M.L., “SICE: an improved missing data
imputation technique,” J. Big Data, vol. 7, no. 37, 2020.
[12] G. Aksu, C. O. Güzeller, and M. T. Eser, “The effect of the
normalization method used in different sample sizes on the success of
Artificial Neural Network model,” Intl. J. of Ass. Tools in Ed., pp.
170–192, 2019.
[13] C. Tantithamthavorn, S. McIntosh, A. E. Hassan, and K. Matsumoto,
“An empirical comparison of model validation techniques for defect
prediction models,” IEEE Trans. on Soft. Eng., vol. 43, no. 1, pp. 1–
18, 2017.
[14] G. Menardi and N. Torelli, “Training and assessing classification rules
with imbalanced data,” Data Mining and Knowledge Discovery, vol.
28, no. 1, pp. 92–122, 2012.
[15] N. Lunardon, G. Menardi, and N. Torelli, “ROSE: a Package for
Binary Imbalanced Learning,” The R Journal, vol. 6, no. 1, p. 79,
2014.
[16] B. Lantz, “Machine learning with R: discover how to build machine
learning algorithms, prepare data, and dig deep into data prediction
techniques with R,” Birmingham: Packt Publishing, 2015.
[17] C. L. Krishna and P. V. Reddy, “Principal Component Analysis on
mixed data for Deep Neural Network classifier in banking
system,” Intl. J. of Comp. Sci. and Eng., vol. 7, no. 5, pp. 129–134,
2019.
[18] M. Buckland and F. Gey, “The relationship between recall and
precision,” J. of the American Society for Inf. Sci., vol. 45, no. 1, pp.
12–19, 1994.
[19] P. Zhang and W. Su, “Statistical inference on recall, precision and
average precision under random selection,” 9th Intl. Conf. on Fuzzy
Sys. And Know. Disc., 2012.
[20] M. Junker, R. Hoch, and A. Dengel, “On the evaluation of document
analysis components by recall, precision, and accuracy,” Proceedings
of the Fifth Intl. Conf. on Doc. Analysis and Rec. (Cat. No.PR00318),
1999.
[21] J. Jeremiah Tanimu, M. Hamada, M. Hassan, and S. Yusuf Ilu, “A
contemporary machine learning method for accurate prediction of
cervical cancer,” SHS Web of Conferences, vol. 102, p. 04004, 2021.