Applied Soft Computing 111 (2021) 107668

Contents lists available at ScienceDirect

Applied Soft Computing

journal homepage: www.elsevier.com/locate/asoc

A predictive intelligence system of credit scoring based on deep

multiple kernel learning✩
∗
Cheng-Feng Wu a,b,c , Shian-Chang Huang d , Chei-Chang Chiou e , , Yu-Min Wang f
a
School of Business Administration, Hubei University of Economics, Wuhan, Hubei Province, China
b
School of Business, Wuchang University of Technology, Wuhan, Hubei Province, China
c
Research Center of Hubei Logistics Development, Hubei University of Economics, Wuhan, Hubei Province, China
d
Department of Business Administration, National Changhua University of Education, Changhua, Taiwan
e
Department of Accounting, National Changhua University of Education, Changhua, Taiwan
f
Department of Information Management, National Chi Nan University, Puli, Taiwan

article info a b s t r a c t

Article history: Banks face the task of improving the accuracy in predicting the behavior of individuals who utilize
Received 7 December 2020 credit cards, as issuing cards to an appropriate applicant is considered an important matter. Credit card
Received in revised form 17 June 2021 debt that is overdue by at least six months has seen a rapid increase in the past decade in the Chinese
Accepted 23 June 2021
credit card industry. The problem of delinquency in credit cards not only affects the development
Available online 7 July 2021
of credit cards but it also influences the sustainability of banks. The use of credit risk assessment is
Keywords: critical in providing a solid foundation upon which the credit card issuer can appropriately approve
Predictive intelligence an applicant for a credit card. In previous studies, a variety of machine learning methods have
Deep learning been proposed to assess credit risk. However, conventional methods are viewed as shallow models
Deep multiple kernel classifier and are not good at representing compositional features. Thus, this study applies a deep multiple
Machine learning kernel classifier as a state-of-the-art technique, which is proficient in coping with deep structure and
Credit scoring complex data in credit risk assessment. It will support decision-makers issuing credit cards in China
appropriately. The results indicate that deep multiple kernel classifier outperforms conventional and
ensemble models. Credit card departments with better risk management can avoid possible bad debt,
hence benefiting banks’ operations. The applications of predictive intelligence enhance the prediction
of human behavior in the credit card industry.
© 2021 Elsevier B.V. All rights reserved.

1. Introduction cards in China has been increasingly frequent in a vast number of

Credit cards have become an increasingly important method
of electronic payment for customers throughout the world. This
is because they offer buyers and vendors several advantages.
These include the use of cash or checks, the provision of reli-
able transaction documents, and documentation of a history of
creditworthiness [1]. In business, providing transaction services
through various payment modes can increase revenue, boost op-
erational efficiency, and minimize operating costs [2]. In previous
studies, forms of electronic payment such as credit card rewards
have been found to increase transparency and accountability and
promote economic growth [3–5].
Although the traditional attitude of ‘‘Save first, Spend later’’
towards money drives Chinese end users [6], the use of credit
✩ Funding: Hubei University of Economics with the grant number
of XJ201901; XJ201902; 11024225. Research Center of Hubei Logistics
Development.
∗ Corresponding author.
E-mail address: ccchiou@cc.ncue.edu.tw (C.-C. Chiou).
transactions. To enhance economic growth after the 2008 reces-
sion, the Chinese government established a long-term strategy for
improving domestic demand by diversifying the emphasis from
exports and investment to domestic consumption [7]. The adop-
tion of credit cards in consumer credit is playing an increasingly
important role in China’s future economic growth. Therefore, an
assessment of credit before credit card approval is vital for the
Chinese credit card industry.
In China, the credit card has reached rapid development with
increasing residential income levels and growing domestic de-
mand. The issuance of credit cards increased rapidly from 0.175
to 0.734 billion from the third quarter of 2009 to the third quarter
of 2019, multiplying more than four times [8].
However, as the amount of credit card holders has increased,
defaults on credit card payments have grown exponentially. Fig. 1
shows credit card debt and balance trends in China over the past
decade. According to the statistics published by the Payment and
Settlement Department of the People’s Bank of China [8], the
amount of credit card debt that is overdue by at least six months
rose by $8.03 billion RMB, from the end of the second quarter

https://doi.org/10.1016/j.asoc.2021.107668
1568-4946/© 2021 Elsevier B.V. All rights reserved.
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al.
Fig. 1. Credit card debt and balance trends in China over the past decade.
of 2009 to $91.92 billion RMB by the third quarter of 2019. This
trend has been increasing sharply for several decades. In partic-
ular, the amount of unpaid credit card balance rapidly increased
by over 33 times, from 0.22 trillion in the third quarter of 2009
to 7.42 trillion RMB in the third quarter of 2019. The problem of
delinquency in credit cards only becomes apparent after credit
cards are issued. This not only affects the development of credit
cards but it also influences the sustainability of banks.
Credit risk assessment is currently critical in providing a solid
foundation upon which the credit card issuer can approve an
applicant for a credit card. Failure to evaluate risk effectively
could result in debt collection, non-performing loans, and an
increasing loss from bad debt, thus threatening the credit card
industry [9–11]. In terms of regulation, banks have been encour-
aged to strengthen their internal model frameworks to achieve
advanced internal accreditation under the Basel II Agreement [12,
13]. Consequently, banks and financial institutions have begun to
thoroughly consider their customers’ credit risk and assess the
efficiency of existing credit risk assessment models.
Because of the importance of credit risk (especially the risk
of delinquency), previous studies have proposed a variety of
data mining methods such as decision trees, logistic regression,
discriminant analysis, and support vector machines (SVMs) [14].
The existing literature has also considered using artificial neural
networks to assess credit risk [15,16]. Over the last decade, deep
learning (DL) (also known as a deep neural network [DNN]) has
emerged as a popular artificial intelligence technique that has
been employed in a wide range of areas. It has performed well
in healthcare and computer science, where decision-makers are
unable to make accurate decisions based on complex but signif-
icant data [17,18]. However, in China, few studies have applied
this novel approach to predict credit card delinquency. Moreover,
what remains unknown is whether this novel approach is supe-
rior to other machine learning approaches in assessing credit card
datasets in China. As the business development in the current
era becomes more complex, the complexity and diversity of data
leads to an increase in the challenges that decision-makers in
financial institutions face while conducting analyses that enable
2
Applied Soft Computing 111 (2021) 107668
them to make robust decisions [19]. Therefore, applying artificial
intelligence techniques, such as DL methods, will assist banks
in assessing applicants’ credit risks and approving credit cards
appropriately.
Taking into account the previous background, the contribution
of this paper is two-fold. First, cardholders’ behavior in east-
ern and western countries differs and only a limited number of
research studies have examined Chinese datasets in the classifica-
tion of ‘‘good credit’’ and ‘‘bad credit’’ applicants. This paper also
focuses on assessing credit card delinquency in the second-largest
economy in the world, where unpaid credit card balance and the
number of applicants has been increasing in recent years. Under
the implementation of Basel II, decisions relating to card approval
are crucial for banks. Second, this study applies a relatively novel
method that is expected to provide a more accurate classification
of card applicants, which will help decrease credit risks in the
issuance of credit cards. This is because traditional statistic meth-
ods might show ineffective performance under complex data. The
relationship between features and the outcome is non-linear or
features only non-monotonous characteristics in credit risk mod-
eling. The weakness of traditional kernel methods is that they use
only a single kernel to represent data. Recent research has shown
that using multiple kernels can effectively enhance the rich-
ness of feature representation and improve performance. Hence,
this study applies a deep multiple kernel classifier (DMKC) as a
state-of-the-art technique that is proficient in coping with deep
structure and complex data in credit risk assessment to support
decision-makers in issuing credit cards in China appropriately.
The remainder of the paper is as follows: Section 2 reviews and
classifies recent literature on credit risk models using conven-
tional and DL methods. Section 3 presents information on kernel
methods and SWMs. Section 4 introduces the multiple kernel
classifier. Section 5 presents the experimental results. Conclu-
sions are then drawn in Section 6.
2. Literature
Because there is a large amount of existing literature based on
the credit risk model, this section’s focus will be on reviewing
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al.
credit risk assessment studies utilizing conventional statistical
and machine learning techniques such as SVMs, neural networks,
logistic regression, discriminant analysis, and DL.
In the literature on SVM-based methods, Xu et al. [20] utilized
a real-world credit dataset to test three-link analysis algorithms
based on the SWM’s preprocessing. They found that methods
for ranking genetic links perform best with regard to classifi-
cation accuracy. Huang et al. [21] employed three methods to
build hybrid SVM-based credit score models to determine each
applicant’s input credit score. Their results showed that SVMs
work better than existing methods of data mining. Harris [22]
compared clustered SVMs with other non-linear SVM-based tech-
niques and found that the clustered SVM yields comparable re-
sults while remaining relatively cheap in computational terms.
Ling and Zhang [23] proposed a new method based on opti-
mization of credit score: the SVM and chaos particle swarm.
Four distinct multi-kernel functions were contrasted with one
radial function kernel. The experimental results showed that the
multi-kernel model is adequate for such a classification problem.
Maldonado et al. [24] proposed two simultaneous classification
and feature selection formulations based on SVMs that explicitly
incorporate acquisition attributes. They also developed mixed-
integer linear programming models for the construction of clas-
sifiers that reduce acquisition costs. The results of an experiment
utilizing datasets for credit scoring demonstrated the efficient
predictive reliability of the methods at a low cost, relative to well-
known feature selection techniques. Luo and Wu [25] applied a
new approach to the issue of credit scoring, involving DL algo-
rithms. In their research, a deep belief network was implemented
with restricted Boltzmann engines, and the classification perfor-
mance was compared with multinomial logistic regression, mul-
tilayer perceptron, and SVM. The results showed that the deep
belief network with restricted Boltzmann machines exceeded
other algorithms. He et al. [26] then developed a three-phase
ensemble model with particle swarm optimization-optimized pa-
rameters that adapted various imbalances to achieve excellent
predictive performance. The results indicated that the average
model performance was superior to comparative algorithms for
different datasets on most evaluation measures. Luo et al. [27]
proposed a new, uninitialized two-stage classification process. A
new kernel-free quadratic surface SVM model was introduced
to prevent selecting kernels and associated kernel parameters;
a golden-sector algorithm was then developed to produce the
required classification for balanced and imbalanced data.
With regard to neural networks, Hsieh and Hung [28] devel-
oped a model that classifies individual neural networks – the
Bayes naïve and the SVM – using class bagging as a data opti-
mization technique for better performance generalization. They
found that the ensemble method significantly improved efficiency
in comparison to traditional ensemble classifications. In another
study, by Zekic-Susac et al. [29], logistic regression, neural net-
work, and decision tree model methods were compared, which
showed that the neural network performs best in prediction
accuracy, followed by the decision tree and the logistic model.
Chen and Huang [30] also developed a scoring model using neural
networks. Subsequently, they used genetic algorithms to provide
advanced knowledge of the class of applicants disqualified by
conditional reclassification. Bekhet and Eletter [31] proposed two
credit scoring models using data mining technologies supporting
Jordanian commercial banks’ loan decisions. The results showed
that the logistic regression model was superior in terms of the
overall accuracy of the radial basis function (RBF) model. By
integrating mathematical and machine learning models, Tsai and
Hung [32] analyzed four approaches and found that hybrid neu-
ronal networks and neural network ensembles outperform single
neural networks. Sohn and Kim [33] proposed a decision-tree-
based credit model that identifies significant predictors of loan
3
Applied Soft Computing 111 (2021) 107668
default for start-up businesses. They compared the performance
of this model with various sets of predictors. The analysis showed
that the proposed model was superior to the logistical regression
model. Xia et al. [34] employed three real-life loan datasets and
two peer-to-peer lending datasets to determine the efficiency of
a newly proposed method that utilized extreme gradient boost-
ing and Bayesian hyper-parameter optimization. The empirical
results showed that Bayesian optimization performed better than
the methods of random search, matrix search, and manual search.
Siswantoro et al. [35] developed a neural network approach in a
Kalman-filter-based linear model. They concluded that the pro-
posed model boosted the efficiency of the neural network. Feng
et al. [36] evaluated a new soft probability, a dynamic ensem-
ble classification method based on ten real-world datasets, and
found that the ability and efficiency of the proposed method can
increase the performance of benchmark predictions. Xia et al. [37]
examined a recent ensemble credit model’s performance that in-
volved six credit scoring datasets along with a bagging algorithm
stacking process. They found that the performance of the stacking
model exceeded that of human calculations and homogeneous
ensemble benchmark models. Zhang et al. [38] evaluated seven
classification methods with five different ensemble methods us-
ing three credit datasets in an ensemble model with a genetic
algorithm. They concluded that the accuracy of the ensemble
classifiers showed improvement following the implementation of
genetic algorithms. Trinkle and Baldwin [39] reviewed previous
studies and concluded that artificial neural networks help com-
panies develop their credit assessment model, hence increasing
their profitability.
Regarding the application of the logit model, Abid et al. [40]
utilized 14 churn datasets to test the logit leaf model. They found
that the current logit leaf model performs better than traditional
predictive approaches. Zhu et al. [41] employed quarterly finan-
cial and non-financial data from listed companies to test three
forms of a two-stage hybrid model composed of logistic regres-
sion and artificial neural network approaches. They concluded
that, in terms of predicting the credit risk of small and medium-
sized enterprises (SMEs), the two-stage hybrid model integrating
a logistic regression and artificial neural network performs better
than the pure neural network method. Ghodselahi [42] tested
the SVM-based hybrid model on a real-world German dataset
using ten clustering and classification techniques. The results
showed that the proposed ensemble model increases classifi-
cation efficiency in the evaluation of credit risk. Khemakhem
and Boujelbene [43] employed real-world data from 86 Tunisian
companies to evaluate discriminant analyses and artificial neural
network methods. They concluded that artificial neural networks
are performing better and are helping firms assess credit risks.
Wang et al. [44] proposed a two-phase hybrid approach based
on the filter methodology and multiple population genetic al-
gorithm to test two UCI database credit-scoring datasets. The
results showed that the proposed model performs better than
the multiple population genetic algorithm. Utilizing a Brazilian
dataset, de Castro et al. [45] assessed various machine learning
algorithms and found that the traditional ensemble model of
bagging, boosting, and random forest was more effective than
other individual classifications.
DL [46], as the name suggests, involves stacking processing
layers one atop the other. The deeper the architecture, the more
layers it has. The intuition behind DL is derived from the com-
positional nature of natural stimuli such as speech and vision.
Natural signals are highly compositional such that simple prim-
itive features combine to form mid-level features and mid-level
features combine to form high-level features. From biochemistry
to genetics, life itself is based on such a compositional structure.
Thus, DL is concerned with learning increasingly more abstract
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al.
representations in a layer-wise manner. Each layer feeds from the
layer below and then sends the output to the layer above. In the
case of vision applications, this process leads to neurons higher up
the hierarchy becoming sensitive to particular complete objects
or scenes.
DL is also useful for unsupervised feature learning. The un-
derlying purpose of DL is to learn a hierarchy of features one
level at a time, which is referred to as greedy layer-wise unsu-
pervised pre-training. This process can be unsupervised, which
means taking advantage of massive unlabeled data for learning
purposes. The learning process begins by training each layer with
unsupervised algorithms (e.g., K-means or restricted Boltzmann
machine [RBM]) and continues by taking the features produced at
the previous level as input for the next layer. Finally, the extracted
features can be used as input for a supervised classifier.
Previous methods generally suffer from two drawbacks: they
rely on a small number of input variables and limited construc-
tion of features. The construction tends to involve a simple math-
ematical operation (not composition), that is, they are types of
shallow models or shallow learning. The training of shallow mod-
els is simple, and they are applicable for only a few features.
Thanks to several recently developed feature-generating methods
(such as DNN), the hierarchical representation of data is available
for a large number of applications. Shallow learning techniques
are, however, poor or inefficient in modeling complex and com-
positional relationships between inputs and outputs. Therefore,
deep architecture is more popular and becomes essential when
high-dimensional data is being processed.
With respect to DL techniques, Pandey [47] utilized a transac-
tion dataset to evaluate DL using the H2O algorithm system. The
results showed that the DL method accurately classifies fraud-
ulent transactions. Galeshchuk and Mukherjee [48] employed
foreign exchange evidence to test the deep convolution of the
neural network. They concluded that the proposed method is
ideal for detecting the change in the direction of foreign ex-
change. Li et al. [49] evaluated approaches based on artificial
intelligence, including neural networks, machine learning, and
DL methods. They found that these methods perform differently
in various datasets; thus, it is essential to find an appropriate
approach for adapting specific data. Sun and Vasarhelyi [50]
employed DNNs to forecast credit card delinquencies. They con-
cluded that it is worthwhile utilizing artificial intelligence as a
method for evaluating credit risk in companies. Wang et al. [51]
applied a DL algorithm to online operation behavior data to assess
default risk. They concluded that, compared with the traditional
artificial feature extraction method, the proposed algorithm sig-
nificantly boosted predictive precision. Yu et al. [52] proposed a
method of forecasting credit risk evaluation using an integrated
learning method composed of an extreme learning machine based
on a multilevel deep trust network. They concluded that the
proposed ensemble learning machine is more efficient than other
conventional ensemble approaches. A novel approach, deep ge-
netic cascade ensembles of classifiers, consisting of two types
of SVM classifiers is proposed in a study by Pławiak et al. [53].
The novel model is used in the banking system to evaluate the
UCI database for the bank credits of the applicants. The system
performs better in classification accuracy than other traditional
ensembles. To get a more comprehensive level of knowledge,
Bastani et al. [54] evaluate the profitability of a credit-scoring
issue as opposed to only evaluating the risk of default. The in-
ternal rate of return as a proxy variable is used to evaluate the
profitability of the peer-to-peer market. This study conducted
a two-stage approach to assess the probability of default to-
gether with the profitability applying a wide and DL strategy. The
proposed method outperforms the existing approaches.
Conversely, kernel methods [55] are effective in learning a
complex decision boundary with only a few parameters. Kernel
4
Applied Soft Computing 111 (2021) 107668
methods have, thus, become popular and successful in numer-
ous applications. Such methods project the data onto a high-
dimensional reproducing kernel Hilbert space, in which simple
linear classifiers can be applied to separate data. However, tra-
ditional kernel methods’ weakness is that they use only a single
kernel to represent data. Recent research has shown that using
multiple kernels can effectively enhance the richness of feature
representation and improve performance [56]. The objective of
multiple kernel learning (MKL) [57] is to learn the kernel from
training data. However, MKL remains a shallow model and is not
good at representing compositional features. Several researchers
have attempted to combine the advantages of both kernel meth-
ods and DL. In so doing, the modified kernel method has become a
new version called deep kernel learning. Cho et al. [58] attempted
to optimize an arc-cosine kernel and successfully integrated this
into a deep architecture. However, in this method, it is difficult
to tune parameters beyond the first layer. In a study conducted
by Zhuang et al. [59], an attempt was made to tune a combi-
nation of kernels. However, optimizing the network beyond two
layers proved difficult. The most important weakness was that
the second layer consisted only of a single Gaussian RBF kernel. To
address these problems, Strobl and Visweswaran [60] developed
a new method capable of optimizing multiple complete layers of
kernels. Their method increased generalization performance on
several types of datasets.
Machine learning models concentrate on predictive precision
and need limited assumptions regarding the characteristic of
the data generation process. This feature allows researchers to
observe the data-driven interactions and non-linear relationships
between explanatory and the outcome variable. Having reviewed
a wealth of studies that have applied statistical and machine
learning techniques, we conclude that it is difficult to determine
the best method for assessing prediction across different datasets.
Indeed, Abdou and Pointon [61] have indicated that no overall
best method for evaluating credit scoring is available. However,
prior studies have indicated that machine learning methods per-
form better than conventional statistical methods, as the rela-
tionships between indicators and the outcome is non-linear or
non-monotone characteristics in credit risk modeling [62]. DL, as
an innovative machine learning method, is rarely used in China
to predict credit card delinquency, which is a major issue in the
current bank industry, and the future credit card industry in China
offers fruitful business opportunities.
3. Previous research
3.1. Kernel methods and support vector machines
The basic principle underlying kernel methods is to map data
from an original input space to a high-dimensional transformed
space, where simple linear classifiers are sufficient to separate
data. SVMs, a type of kernel method, were first proposed by
Vapnik [55]. SVM classifiers construct a hyperplane in high-
dimensional transformed space to separate two classes (labeled
y ∈ {−1, 1}) so that the margin (the distance between the
decision hyperplane and the nearest point) is maximal. This
is based on the structured risk minimization (SRM) principle.
Instead of minimizing the empirical error, the advantage of SVMs
is that they minimize an upper bound of the generalization (or
SRM) error. Minimizing empirical error is usually the case in other
neural networks. The SVM classification function is formulated as
follows:
y = sign(wT φ (x) + b), (1)
where φ (x) is the feature function, which denotes nonlinear map-
ping used to map data from the input space x to the future space.
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al. Applied Soft Computing 111 (2021) 107668

The coefficients w and b are estimated based on the generaliza- where α, β, γ stand for the free parameters of the polynomial
tion error using the following optimization problem: kernel. Similarly, the K(2) of a Gaussian RBF kernel is
K(2) = Φ (2) Φ (1) (x) · Φ (2) Φ (1) (y)
l
( ) ( )
1 ∑ (13)
min R(w, ξ ) = ∥w∥ + C2
ξi , (2) −2γ (1−K(1) (x,y))
w,b 2
i=1
=e . (14)

with Using basic functions, kernels can be designed to create differ-

ent representations. To increase representation power (or rich-
yi (w φ (xi ) + bi ) ≥ 1 − ξi , i = 1, . . . , l
T
(3) ness) in a DL structure, we can stack two kernels of different
ξi ≥ 0, i = 1, . . . , l, (4) types; this develops a different representation from either alone.
The most important aspect to consider is that a single kernel does
where C is a prescribed parameter used to determine the trade- not easily approximate the richer representations of multiple
off between the empirical risk and model smoothness (a smooth kernels.
and simple function is the best choice). However, how can we measure the richness of a data repre-
Based on convex analysis, we can take the Lagrangian function sentation? Strobl and Visweswaran [60] proposed a method to
and pose an optimal condition for the above problem. The dual analyze the richness/complexity of a kernel based on its pseudo-
solution of this problem can be formulated as follows: dimension. More specifically, they measured richness/complexity
l l by the upper bound of the second-order Rademacher chaos com-
∑ 1∑
max D(α ) = αi − yi yj αi αj K (xi , xj ), (5) plexity [63]. Using their measure, multiple layers (or deep ar-
α 2
i=1 i,j=1 chitecture) can increase the representation richness of kernels.
However, the increased richness of the kernels can also increase
with constraints,
the risk of over-fitting. Nevertheless, compared with traditional
0 ≤ αi ≤ C , i = 1, . . . , l (6) DNNs, Strobl and Visweswaran [60] demonstrated that the upper
l bound of the generalization error for DMKC is significantly less
than that of deep feedforward networks.
∑
αi yi = 0, (7)
The architecture of DMKC used in this study is an l-level
i=1
multiple kernel architecture with h sets of m kernels at each
where α are Lagrangian multipliers, and K (xi , xj ) is the ker- layer:
nel function. The solution of b can follow the complementarity { ( ) }
(l) (l) (l) (l)
K(l) = W1,1 K1,1 W1,1 K1,1 + · · · + · · · + Wh,m Kh,m ,
(l−1) (l−1)
Karush–Kuhn–Tucker (KKT) conditions. Finally, the classification (15)
function is given by
(l)
( l
) where Kh,m represents the mth kernel in set h at layer l with an
(l)
associated weight parameter Wh,m , and K(l) represents the single
∑
f (x) = sign αi yi K (x, xi ) + b . (8)
i=1
combined kernel at layer l.
The most important is the kernel function, which is equal to the 5. Experimental results
inner product of two vectors x and xi in the feature space, namely,
K (x, xi ) = φ (x)φ (xi ). To be a kernel function, it must satisfy 5.1. Data sources and sampling
Mercer’s condition. For details, please refer to Vapnik [55].
The sample comprised a Chinese credit dataset obtained from
4. Deep Multiple Kernel Classifier (DMKC) the big data platform of machine learning databases (www.cfdsj.
cn). It contained financial information on a large number of cus-
The most critical issue in kernel methods is data representa- tomers of an anonymized bank. Of these, 517 were creditworthy
tion, which is implicitly chosen through the so-called kernel. Re- cardholders and 183 exhibited credit card delinquency. This data
cent applications have shown that using multiple kernels rather set included one categorical attribute: education, and seven nu-
than one can enhance the richness of feature representation and merical attributes: age, seniority, duration of residence, income,
improve performance [56]. Recently, Strobl and Visweswaran [60] debt to income ratio, credit card debt, and other debt.
proposed a deep version of the multiple kernel classifier. They
combined kernels at each layer and successfully optimized the DL 5.2. Experimental results
structure. Their experiments on many datasets have shown that
each layer can increase performance with just a few base kernels. Customer credit ratings provide important information on
The role of kernels within kernel methods is used to compute credit risk for banks in financial markets. DL was, therefore,
the similarity between two inputs. Traditionally, the dot product applied to create a novel rating prediction system. The perfor-
of its two basic functions can be used to describe a kernel mance of the system was then examined in relation to the sample
K(1) (x, y) = Φ (1) (x) · Φ (1) (y), (9) dataset.
Descriptive statistics for the data are listed in Table 1 for refer-
(1)
where K (x, y) represents a first-layer kernel. In DL (or com- ence. In the first stage, we tested several traditional models used
positional) structure, how to view a kernel within a kernel is as to forecast credit ratings, namely, multilayer perceptron (MP),
respective basic functions within basic functions; for example, an Bayesian network (BayesNet), SVM, nearest neighbors classifier
l number of layers (IBK), logistic regressions, and decision tree (J48). In the second
K(l) (x, y) = Φ (l) . . . Φ (1) (x) · Φ (l) . . . Φ (1) (y) ,
( ) ( )
(10) stage, we compared several advanced ensemble models, namely,
AdaBoost M1, bagging, stacking, random committee, and random
For example, as for the polynomial kernel, a polynomial of higher subspace. The base classifiers of these ensemble methods were
order can be formed by the following operation: set up as a decision tree (J48), except for the random committee,
which used a random tree as base classifiers. The dataset was
K(1) (x, y) = (α (x · y) + β )γ (11)
randomly divided into ten parts, and ten-folds cross validation
(2) (1) γ
K (x, y) = α (K (x, y) + β ) , (12) was applied to evaluate the models’ performance.
5
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al. Applied Soft Computing 111 (2021) 107668

Fig. 2. The performance comparison of traditional models.

Fig. 3. The performance comparison of advanced models.

Table 1
Descriptive statistics of input variables.
Age Education Seniority Duration of Residence Income Debt-to-Income Ratio Credit Card Debt Other Debt
Min. 20 1 0 0 14 0.4 0.011696 0.045584
Max. 56 5 31 34 446 41.3 20.56131 27.0336
Median 34 1 7 7 34 8.6 0.85487 1.987568
Mode 29 1 0 2 25 4.5 0.085785 7.8234
Range 36 4 31 34 432 40.9 20.54961 26.98802
Std. 7.997342 0.928206 6.658039 6.824877 36.81423 6.827234 2.117197 3.287555
Skewness 0.360703 1.198744 0.829371 0.936086 3.850475 1.093713 3.890258 2.722314

Traditionally, multiple kernel learning algorithms used RBF Table 2

and polynomial kernels. Since our objective is to maximize the Performance comparison of traditional models.

upper bound of the pseudo-dimension of the final kernel (which BayesNet MP Logistic SVM IBK (with 3 neighbors) J48

will increase its richness with each successive layer), this study 75.14% 77.29% 80.43% 79.29% 77.14% 75.71%
chooses not to use these traditional kernels. This study used four Note: BayesNet denotes Bayesian network; MP denotes multilayer perceptron;
unique base kernels: a linear kernel, an RBF kernel, a polynomial Logistic denotes logistic regressions; SVM denotes support vector machines; IBK
kernel of degree 2, and a polynomial kernel of degree 3. We used denotes nearest neighbors classifier; J48 denotes decision tree.

one set of kernels for each layer and made a 5-layer architecture
for the DMKC. This study used a gradient descent algorithm to
train the DMKC. these ensemble models. Although ensemble models usually per-
The accuracies (in %) of traditional models are presented in form well in many applications and are robust, they do not appear
Table 2 and Fig. 2. The best was logistic regression, followed to be very effective in this case. This may be attributed to the
by SVM. These were better than other classifiers. The poorest
fact that although ensemble methods are powerful, their base
in terms of performance was the BayesNet. We then compared
several advanced ensemble models with our DMKC. Details on the classifiers are shallow models and cannot take full advantage
comparative performance of these advanced models is presented of ensemble methods. If we utilize deep models as the base
in Table 3 and Fig. 3, which show that our DMKC outperforms classifiers for an ensemble model, the computation required will
6
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al.
Table 3
Performance comparison of advanced models.
DMKC AdaBoostM1 Bagging
82.86% 75% 77.5714%
Note: DMKC denotes deep multiple kernel classifier.
be extreme, which is not affordable on a general computation
platform.
Nonlinear kernel methods are prevalent in financial data min-
ing. The kernel is effective at discovering nonlinear patterns in
data. Customer credit data is usually complex and have a deep
structure. Traditional kernel methods cannot fully capture or
represent complex, compositional, and hierarchical data features.
Consequently, this study applied DMKC to handle customer credit
data. Compared with the ensemble and traditional methods, the
DMKC exhibited the best performance. The computational load-
ing was not as heavy as that of an ensemble model with deep
base classifiers. Therefore, the framework proposed by this study
is both effective and efficient for this type of customer credit data.
6. Conclusion
Credit risk assessment is a widely employed method that helps
credit card departments decide whether to approve credit card
applicants. Machine learning techniques are utilized in credit risk
assessment as customer credit data is complex and has a deep
structure. A DL method such as the DMKC employed in this study
performs better than other traditional statistic methods when
certain essential assumptions cannot be satisfied for complex
data.
This experiment shows that the deep kernel-based method in
machine learning is relatively effective in this Chinese dataset
compared to traditional statistical approaches such as BayesNet,
Logistic, IBK, and J48. This finding contributes knowledge on
Chinese applicants to the literature on credit risk assessment for
credit card approval, a field in which datasets in Western coun-
tries, such as UCI Machine Learning Repository, have been applied
in the vast majority of studies. The results in this paper are similar
to those of Li et al. [64], who focused on Australian and German
credit card datasets and found that the deep kernel-based method
performs better than traditional statistical approaches. Regarding
deep multiple kernel learning, Huang and Wu [65] indicated deep
kernel-based method is more effective in forecasting energy price.
Additionally, testing on 12 real-world databases, Rebai et al. [66]
addressed that deep kernel-based method better obtains the ac-
curacy of classification. Finally, this study provides evidence to
show that the DMKC performs relatively better than traditional
methods in shallow learning, as customer credit data are complex
with a deep structure. The effectiveness of DMKC is derived from
the fact that it combines multiple kernels within each layer to
increase the richness of representations and stacks various layers
to process a signal in an increasingly abstract manner.
The better the accuracy of classification, the lower the risk of
operations in the credit card business. Banks using DL methods
can adjust the number of credit lines for customers holding
credit cards who are probably going into delinquency. Credit card
departments with better risk management can avoid possible bad
debt, which will benefit banks’ operations [67], especially those
in the world’s second-largest economy, where the unpaid credit
card balance has rapidly increased by over 33 times in recent
decades. The strength of DL shows that it contribute more to
financial institutions as the available data is large and compli-
cated. In addition, an efficient model is expected to use powerful
algorithms to assess credit scoring automatically [50]. Similarly,
Blumenstock et al. [68] examine machine learning techniques
with DL methods for survival analysis in a credit risk modeling
Applied Soft Computing 111 (2021) 107668
Stacking RandomCommittee RandomSubspace
73.8571% 74.71% 77.43%
framework. The findings offer clear proof that DL models gener-
ate more reliable performance on default and prepayment risk
than conventional models. Based on this observation, analysts
assessing the sustainability of a business could regard DL meth-
ods as standards for validating other frameworks such as the
International Financial Reporting Standard 9 model.
Thus, constructing a better credit risk assessment model to
classify good and bad credit card holders is an important and
essential task for bank managers and policy-makers in China.
This research’s future work is to build a hybrid classification
process focused on further features to assess a credit risk model.
In addition, large-scale applications of other credit risk datasets
are expected in the future.
CRediT authorship contribution statement
Cheng-Feng Wu: Research topic, Data collection, Developed
implications, Writing. Shian-Chang Huang: Research model and
statistical analysis, Developed implications, Writing. Chei-Chang
Chiou: Statistical analysis, Edited this manuscript, Developed im-
plications, Responsible for correspondence. Yu-Min Wang: Statis-
tical analysis, Developed implications.
Declaration of competing interest
The authors declare that they have no known competing finan-
cial interests or personal relationships that could have appeared
to influence the work reported in this paper.
References
[1] J.B. Soll, R.L. Keeney, R.P. Larrick, Consumer misunderstanding of credit
card use, payments, and debt: Causes and solutions, J. Publ. Policy Mark.
32 (1) (2013) 66–81.
[2] S.C. Alliance, Contactless Payment and the Retail Point of Sale: Applications,
Technologies and Transaction Models, Smart Card Alliance, Princeton
Junction, NJ, 2003.
[3] O.S. Oyewole, J.G. El-Maude, M. Abba, Onuh. M.E., Electronic payment
system and economic growth: a review of transition to cashless economy
in Nigeria, Int. J. Sci. Eng. Technol. 2 (2013) 913–918.
[4] M. Zandi, V. Singh, J. Irving, The impact of electronic payments on eco-
nomic growth. Moody’s Analytics: Economic and Consumer Credit Analyt-
ics, 217 (2), , 2013, https://www.visa.co.in/content/dam/vcom/download/
corporate/media/moodys-economy-white-paper-feb-2013.pdf.
[5] E.G. Mieseigha, U.K. Ogbodo, An empirical analysis of the benefits of
cashless economy on Nigeria’s economic development, J. Finance Account.
4 (2013) 11–16.
[6] L. Wang, W. Lu, N.K. Malhotra, Demographics, attitude, personality and
credit card features correlate with credit card debt: a view from China, J.
Econ. Psychol. 32 (1) (2011) 179–193.
[7] N. Ding, Consumer credits and economic growth in China, Chin. Econ. 48
(4) (2015) 269–278.
[8] Payment and settlement department of the people’s bank of China, China
payment system, 2020, http://www.pbc.gov.cn/zhifujiesuansi/128525/
128545/128643/17694/index1.html.
[9] B. Twala, Multiple classifier application to credit risk assessment, Expert
Syst. Appl. 37 (4) (2010) 3326–3336.
[10] S.C. Chen, M.Y. Huang, Constructing credit auditing and control & manage-
ment model with data mining technique, Expert Syst. Appl. 38 (5) (2011)
5359–5365.
[11] S.C. Huang, C.F. Wu, Customer credit quality assessments using data
mining methods for banking industries, Afr. J. Bus. Manage. 5 (11) (2011)
4438–4445.
[12] BCBS, International Convergence of Capital Measurement and Capital
Standards: A Revised Framework- Comprehensive Version, Bank for
International Settlements, 2006.
7
C.-F. Wu, S.-C. Huang, C.-C. Chiou et al.
[13] BIS, Implementation of Basel II: Practical Considerations, Bank for
International 919 Settlements, 2004.
[14] A.I. Marques, V. Garcia, J.S. Sanchez, Exploring the behaviour of base
classifiers in credit scoring ensembles, Expert Syst. Appl. 39 (11) (2012)
10244–10250.
[15] H.C. Koh, K.L.G. Chan, Data mining and customer relationship mar- keting
in the banking industry, Singapore Manage. Rev. 24 (2) (2002) 1–27.
[16] L.C. Thomas, A survey of credit and behavioural scoring: forecasting
financial risk of lending to consumers, Int. J. Forecast. 16 (2) (2000)
149–172.
[17] P. Hamet, J. Tremblay, Artificial intelligence in medicine, Metabolism 69S
(2017) S36–S40.
[18] C. Ohlsson, Exploring the Potential of Machine Learning: How Ma-
chine Learning Can Support Financial Risk Management (Master’s thesis),
Uppsala University, 2017.
[19] E. Turban, J.E. Aronson, T.-P. Liang, R. McCarthy, in: A.K. Ghosh (Ed.), De-
cision Support Systems and Intelligent Systems, seventh ed., Prentice-Hall
of India, New Delhi, India, 2005.
[20] X. Xu, C. Zhou, Z. Wang, Credit scoring algorithm based on link analysis
ranking with support vector machine, Expert Syst. Appl. 36 (2) (2009)
2625–2632.
[21] C.L. Huang, M.C. Chen, C.J. Wang, Credit scoring with a data mining
approach based on support vector machines, Expert Syst. Appl. 33 (4)
(2007) 847–856.
[22] T. Harris, Credit scoring using the clustered support vector machine, Expert
Syst. Appl. 42 (2) (2015) 741–750.
[23] Y. Ling, Q. CAO, H. Zhang, Credit scoring using multi-kernel support vector
machine and chaos particle swarm optimization, Int. J. Comput. Intell. Appl.
11 (03) (2012) 1250019.
[24] S. Maldonado, J. Pérez, C. Bravo, Cost-based feature selection for Support
Vector Machines: An application in credit scoring, European J. Oper. Res.
261 (2) (2017) 656–665.
[25] C. Luo, D. Wu, D. Wu, A deep learning approach for credit scoring using
credit default swaps, Eng. Appl. Artif. Intell. 65 (2017) 465–470.
[26] H. He, W. Zhang, S. Zhang, A novel ensemble method for credit scoring:
Adaption of different imbalance ratios, Expert Syst. Appl. 98 (2018)
105–117.
[27] J. Luo, X. Yan, Y. Tian, Unsupervised quadratic surface support vector
machine with application to credit risk assessment, European J. Oper. Res.
280 (3) (2020) 1008–1017.
[28] N.C. Hsieh, L.P. Hung, A data driven ensemble classifier for credit scoring
analysis, Expert Syst. Appl. 37 (1) (2010) 534–545.
[29] M. Zekic-Susac, N. Sarlija, M. Bensic, Small business credit scoring: a com-
parison of logistic regression, neural network, and decision tree models,
in: 26th International Conference on Information Technology Interfaces,
IEEE, 2004, pp. 265–270.
[30] M.C. Chen, S.H. Huang, Credit scoring and rejected instances reassigning
through evolutionary computation techniques, Expert Syst. Appl. 24 (4)
(2003) 433–441.
[31] H.A. Bekhet, S.F.K. Eletter, Credit risk assessment model for Jordanian
commercial banks: neural scoring approach, Rev. Dev. Finance 4 (1) (2014)
20–28.
[32] C.F. Tsai, C. Hung, Modeling credit scoring using neural network ensembles,
Kybernetes 43 (7) (2014) 1114–1123.
[33] S.Y. Sohn, J.W. Kim, Decision tree-based technology credit scoring for
start-up firms: Korean case, Expert Syst. Appl. 39 (4) (2012) 4007–4012.
[34] Y. Xia, C. Liu, Y. Li, N. Liu, A boosted decision tree approach using Bayesian
hyper-parameter optimization for credit scoring, Expert Syst. Appl. 78
(2017) 225–241.
[35] J. Siswantoro, A.S. Prabuwono, A. Abdullah, B. Idrus, A linear model based
on Kalman filter for improving neural network classification performance,
Expert Syst. Appl. 49 (2016) 112–122.
[36] X. Feng, Z. Xiao, B. Zhong, J. Qiu, Y. Dong, Dynamic ensemble classification
for credit scoring using soft probability, Appl. Soft Comput. 65 (2018)
139–151.
[37] Y. Xia, C. Liu, B. Da, F. Xie, A novel heterogeneous ensemble credit scoring
model based on bstacking approach, Expert Syst. Appl. 93 (2018) 182–199.
[38] H. Zhang, H. He, W. Zhang, Classifier selection and clustering with fuzzy
assignment in ensemble model for credit scoring, Neurocomputing 316
(2018) 210–221.
[39] B.S. Trinkle, A.A. Baldwin, Research opportunities for neural networks:
the case for credit, Intell. Syst. Account. Finance Manage. 23 (3) (2016)
240–254.
Applied Soft Computing 111 (2021) 107668
[40] L. Abid, A. Masmoudi, S. Zouari-Ghorbel, The consumer loan’s payment
default predictive model: an application of the logistic regression and the
discriminant analysis in a Tunisian commercial bank, J. Knowl. Econ. 9 (3)
(2018) 948–962.
[41] Y. Zhu, C. Xie, B. Sun, G.J. Wang, X.G. Yan, Predicting China’s SME credit risk
in supply chain financing by logistic regression, artificial neural network
and hybrid models, Sustainability 8 (5) (2016) 433.
[42] A. Ghodselahi, A hybrid support vector machine ensemble model for credit
scoring, Int. J. Comput. Appl. 17 (5) (2011) 1–5.
[43] S. Khemakhem, Y. Boujelbene, Credit risk prediction: A comparative study
between discriminant analysis and the neural network approach, Account.
Manage. Inf. Syst. 14 (1) (2015) 60.
[44] D. Wang, Z. Zhang, R. Bai, Y. Mao, A hybrid system with filter approach
and multiple population genetic algorithm for feature selection in credit
scoring, J. Comput. Appl. Math. 329 (2018) 307–321.
[45] J.R. de Castro Vieira, F. Barboza, V.A. Sobreiro, H. Kimura, Machine learning
models for credit analysis improvements: Predicting low-income families’
default, Appl. Soft Comput. 83 (2019) 105640.
[46] Y. Bengio, Y. LeCun, G. Hinton, Deep learning, Nature 521 (2015) 436–444.
[47] Y. Pandey, Credit card fraud detection using deep learning, Int. J. Adv. Res.
Comput. Sci. 8 (5) (2017).
[48] S. Galeshchuk, S. Mukherjee, Deep networks for predicting direction of
change in foreign exchange rates, Intell. Syst. Account. Finance Manage.
24 (4) (2017) 100–110.
[49] Y. Li, W. Jiang, L. Yang, T. Wu, On neural networks and learning systems
for business computing, Neurocomputing 275 (2018) 1150–1159.
[50] T. Sun, M.A. Vasarhelyi, Predicting credit card delinquencies: An application
of deep neural networks, Intell. Syst. Account. Finance Manage. 25 (4)
(2018) 174–189.
[51] C. Wang, D. Han, Q. Liu, S. Luo, A deep learning approach for credit scoring
of peer-to-peer lending using attention mechanism LSTM, IEEE Access 7
(2018) 2161–2168.
[52] L. Yu, Z. Yang, L. Tang, A novel multistage deep belief network based
extreme learning machine ensemble learning paradigm for credit risk
assessment, Flexible Serv. Manuf. J. 28 (4) (2016) 576–592.
[53] P. Pławiak, M. Abdar, U.R. Acharya, Application of new deep genetic
cascade ensemble of SVM classifiers to predict the Australian credit
scoring, Appl. Soft Comput. 84 (2019) 105740.
[54] K. Bastani, E. Asgari, H. Namavari, Wide and deep learning for peer-to-peer
lending, Expert Syst. Appl. 134 (2019) 209–224.
[55] V.N. Vapnik, The Nature of Statistical Learning Theory, second ed., Springer,
New York, 1999.
[56] M. Varma, B.R. Babu, More generality in efficient multiple kernel learning,
in: Proceedings of the International Conference on Machine Learning,
Montreal, Canada, 2009, pp. 1065–1072.
[57] G.R.G. Lanckriet, N. Cristianini, L.E. Ghaoui, P. Bartlett, M.I. Jordan, Learning
the kernel matrix with semidefinite programming, J. Mach. Learn. Res. 5
(2004) 27–72.
[58] Y. Cho, S.K. Saul, Kernel methods for deep learning, Adv. Neural Inf.
Process. Syst. 22 (2009) 342–350.
[59] J. Zhuang, I.W. Tsang, S.C.H. Choi, Two-layer multiple kernel learning,
in: Proceedings of International Conference on Artificial Intelligence and
Statistics, 2011, 2011.
[60] E.V. Strobl, S. Visweswaran, Deep multiple kernel learning, in: 2013 12th
International Conference on Machine Learning and Applications, Vol. 1,
IEEE, 2013, pp. 414–417.
[61] H.A. Abdou, J. Pointon, Credit scoring, statistical techniques and evaluation
criteria: a review of the literature, Intell. Syst. Account. Finance Manage.
18 (2–3) (2011) 59–88.
[62] M. Moscatelli, F. Parlapiano, S. Narizzano, G. Viggiano, Corporate default
forecasting with machine learning, Expert Syst. Appl. 161 (2020) 113567.
[63] Y. Ying, C. Campbell, Rademacher chaos complexities for learning the
kernel, Neural Comput. 22 (2010) 2858–2886.
[64] J. Li, L. Wei, G. Li, W. Xu, An evolution strategy-based multiple kernels
multi-criteria programming approach: The case of credit decision making,
Decis. Support Syst. 51 (2) (2011) 292–298.
[65] S.C. Huang, C.F. Wu, Energy commodity price forecasting with deep
multiple kernel learning, Energies 11 (11) (2018) 3029.
[66] I. Rebai, Y. BenAyed, W. Mahdi, Deep multilayer multiple kernel learning,
Neural Comput. Appl. 27 (8) (2016) 2305–2314.
[67] F. Butaru, Q. Chen, B. Clark, S. Das, A.W. Lo, A. Siddique, Risk and
risk management in the credit card industry, J. Bank. Financ. 72 (2016)
218–239.
[68] G. Blumenstock, S. Lessmann, H.V. Seow, Deep learning for survival and
competing risk modelling, J. Oper. Res. Soc. (2020) 1–13.