Indian Journal of Fibre & Textile Research

Vol. 39, March 2014, pp. 55-59

Performance of SVM based on PUK kernel in comparison to SVM based on RBF

kernel in prediction of yarn tenacity
Khalid AA Abakar & Chongwen Yua
College of Textiles, Donghua University, Shanghai 201620, People's Republic of China

Received 25 May 2012; revised received and accepted 23 January 2013

A new kernel function of SVM based on the Pearson VII function has been applied and compared with the commonly
applied kernel functions, i.e. the polynomial and radial basis function (RBF), to predict yarn tenacity. It is found that the
SVM model based on Pearson VII kernel function (PUK) shows the same applicability, suitability, performance in
prediction of yarn tenacity as against SVM based RBF kernel. The comparison with the ANN model shows that the two
SVM models give a similar predictive performance than ANN model.

Keywords: Artificial neural network, Pearson VII kernel function (PUK) kernel, Radial basis function kernel, Support
vector machines, Yarn properties

1 Introduction linear form. Some popular kernels are linear,

Modeling of yarn properties and the relationship
between fibre and yarn properties are the most
fascinating areas of research in the domain of textile
engineering. Researchers have made a lot of efforts to
formulate mathematical, statistical, empirical and
intelligent models to predict various yarn properties,
such as yarn strength and yarn unevenness. In recent
years, there has been growing use of soft computing
tools or intelligent models like artificial neural
network (ANN)1-3, fuzzy logic4-5 and genetic
algorithm or genetic programming (GP)6 to predict
various yarn properties and to optimize the process.
Most recently, support vector machines (SVMs) is
one of the best machine learning or data mining
techniques of knowledge discovery that aims at
extracting the information from databases. The
suitability of using SVM in the prediction of cotton
yarn properties to know the accuracy of prediction is
studied by many reviews7-9.
They found that like ANN model, the SVM model
is able to predict with a reasonably good accuracy in
most cases. However, one of the main reasons for the
popularity of SVM is its ability to model complex
nonlinear relationships by selecting a suitable kernel
function. Briefly, the kernel function transforms the
input space into a high dimensional feature space
where non-linear relationships can be represented in a
kernel in comparison to the commonly applied SVM
a
Corresponding author.
E-mail: yucw@dhu.edu.cn
polynomial, Gaussian (radial basis function (RBF))
and Sigmoid kernel. The particular choice of a kernel
function to map the non-linear input space into a
linear feature space depends highly on the nature of
the data, i.e. which kind of underlying relationship
needs to be estimated to relate the input data with the
desired output property. Because the nature of the
data is usually unknown, the best mapping function
must be determined experimentally by applying and
validating various kernel functions yielding the
highest generalization performance. Therefore, it will
be very useful to search a kind of universal kernel,
which can be adaptive for many types of data through
adjusting the kernel parameters.
Pearson VII Universal Kernel was applied as a
kernel function of SVM10, and referred to as PUK.
The Pearson VII function has excellent flexibility and
possibility to change easily, from a Gaussian into a
Lorentzian peak shape and more by adapting its
parameters. So, it is possible to use the Pearson VII
function as a generic kernel which can replace the
earlier mentioned set of kernel functions.
In this work, attempt has been made to study the
nonlinear regression problem of fibre/ yarn properties
relationships, and to investigate the applicability,
suitability and performance of SVM based on PUK
based on the RBF kernel to predict yarn tenacity from
cotton fibre properties.
56 INDIAN J FIBRE TEXT RES, MARCH 2014

2 Materials and Methods where ( x ) is the features of the input variables,

while w and b are the coefficients and they estimated
2.1 Data Set by minimizing
The cotton fibre properties and corresponding yarn
tenacity data were collected from the published 1 N
1

2
E ( w) = C yi f ( xi , w) + w (2)
review11, using the case-based reasoning technique to N i =1 2
predict yarn properties. A total of twenty-five
different cotton samples were collected and measured 1 N
by a high volume instrument (HVI). The fibre
The first term yi f ( xi , w) is the
N i =1
C
properties used as input data were fibre strength
empirical risk (error) and can be given by the
(g/tex), fibre length (inch), elongation (%), trash
insensitive loss function, as shown bwlow:
content (Cnt), length uniformity (%), and micronaire
(M). The yarn tenacity (cN/tex) was used as the 0 if x <
target. A set of twenty-five samples was used to train x := (3)
and test the models. x else

2.2 Introduction to Support Vector Machines 1 2

The term w is the regularization term and is the
Support vector machines (SVMs) introduced by 2
Vapnik12 are machine learning methods based on measure of the function fitness. The value of C ,
statistic theory. It was proposed as a novel approach being the a regularization constant , determines the
for solving classification problems. Additionally, with trade-off between the empirical risk and the
the introduction of Vapniks -insensitive loss regularization term.
function, the SVM has been extended to solve a In order to obtain the estimations w and b , Eq. (2)
nonlinear regression- estimation problem, called the is transformed into following Eq. (4) as optimal
SVM for regression (SVR)13,14. function, by introducing the positive slack variables
Recently, the SVR has been applied to various i and i* as follows:
fields such as optimal control, time-series prediction
and interval regression analysis15-17. However, to n
1
Minimize E ( w) = C ( i + i* ) +
2
obtain better performance, some parameters (called w
i =1 2
hyperparameters) must be selected carefully in the
SVM or SVR18,19. The hyperparameters include the Subjected to
kernel parameters, the epsilon value in Vapniks -
insensitive loss function, and the regularization yi f ( xi , w) + i
constant (The complexity parameter C). *
f ( xi , w) yi + i (4)
Basic comprehensive description of the concept *
underlying SVR modeling can be given as i , i 0
considering a data set {( x1 , y1 ), ( xi , yl )} ,
Slack variables i and i* can be introduced when
where x R d is the dimensional input space and
data cant be estimated by the function f under the
y R is the corresponding output and it can be
precise . Introducing Lagrange multipliers and
written as a regression function y = f ( x ).
according to Karush-Kuhn-Tucker conditions, Eq. (4)
The basic idea of support vector regression is to can be transformed into the following form:
map the input vector x into high dimensional feature
space by nonlinear mapping function and then to Minimize:
perform linear regression in the feature space. N l
However, the correlation between input and output LP ( * , ) = ( i* + i ) + yi ( i* i ) +
can be written as the following formula: i =1 i =1
(5)
1 N N

f ( x ) = w. ( x ) + b (1)

2 i =1 j =1
( i* i )( *j i ) K ( xi , x j )
 ABAKAR & YU: SVM BASED ON PUK KERNEL IN COMPARISON TO SVM BASED ON RBF KERNEL
Subjected to:
N
(
i =1
*
i i ) = 0,
* kind of kernel function of SVM was chosen wrongly.
0 , i C , i = 1, 2,..., N
i The pearson VII kernel function of multi-dimensional
In Eq. (6), i , i* are Lagrange multipliers. The
model output is given below:
N (11)
f ( x, ) = ( i* i )K ( xi , x j ) + b. (7)
i =1
where K(.,.) is the kernel function. Kernel function
can have various forms but the two widely used
kernel for real -valued data are polynomial and
Gaussian radial basis function (RBF) kernel. The
polynomial kernel of degree d is defined as
K ( xi , x j ) = ( xi .x j + k ) d
where k is the constant. The kernel with d = 1 is the
linear kernel function. The second very widely used
kernel is the Gaussian radial basis function (RBF)
kernel defined by
K ( xi , x j ) = exp( xi , x j / 2 ) 2 (9)
where > 0 is a parameter that controls the width of
the Gaussian. It plays a similar role as the degree of
the polynomial kernel in controlling the flexibility of
the resulting classifier.
Pearson VII universal kernel (PUK) is the other
type of kernel function that can be used in support
vector machines and based on Ustun et al.10. The
general form of the Pearson VII function for curve
fitting purposes is given by the following relationship:
f ( x) = H /[1 + (2( x x0 ) 2(1/ ) 1 / ) 2 ]
(10)
where H is the peak height at the center x0 of the
peak, and x represents the independent variable. The
parameters and control the half-width (also
named Pearson width) and the tailing factor of the
peak. The main reason to use the Pearson VII function
for curve fitting is its flexibility to change, by varying
the parameters and .
Pearson VII function is adopted as an alternative
generic kernel function in this study. It might serve as
a kind of universal kernel which can replace
(by selecting the appropriate parameter setting) the set
57
of commonly applied kernel functions, i.e. the linear,
polynomial, Gaussian and Sigmoid kernels. Adopting
(6)
Pearson VII function as kernel function, it might
avoid the case that SVM cant match data well if the
input space is given by the following formula:
2
K ( xi , x j ) = 1/[1 + (2 xi x j 2(1/ ) 1 / )2 ]
There are no examples of application of PUK kernel
methods available in textile engineering related
research despite the many advantages that it
offers.According to Eq. (10), Pearson VII universal
kernel function, referred to as PUK in this study, is
used as SVM kernel function and also compared to
the commonly used kernel the polynomial and
(8) Gaussian radial basis (RBF) and ANN.
2.3 Artificial Neural Network
Artificial neural network (ANN) is suitable for
modeling nonlinear relationship. It is a powerful data
modeling tool that is able to capture and represent any
kind of input output relationships. The theory of
ANN and its application in textile engineering exactly
in spinning process and modeling yarn properties
studies have extensively been discussed. The
ANN employed in this study is a three layer back-
propagation (BP) multilayer perceptron network, the
input layer, one hidden layer, and the output layer.
More details on the theory and applications of the
ANN can be found in a number of publications1-3.
2.4 Implementation
The parameters of support vector machine for
regression (SVMR) such as the complexity parameter
C, the values of - insensitive loss function , and the
kernel parameters such as degree d of polynomial
kernel, width of RBF kernel function , and PUK
kernel parameters ( and ) were optimized by using
grid search approach, in the training set data using
10-fold cross validation. This grid search tries values
of each parameter across the specified search range
using geometric steps. This approach tries to search
best possible values using each parameters across
specified chosen search range, hence the best
performance results were reported with good
prediction accuracy.
To evaluate the prediction performance of each
algorithm, a 10-fold cross validation technique is
58 INDIAN J FIBRE TEXT RES, MARCH 2014
used. In 10-fold cross validation, the initial dataset is
randomly partitioned into 10 mutually exclusive
subsets or folds D1, D2, , D10, each of
approximately equal size. The training and testing are
performed 10 times. In iteration i, partition Di is
reserved as the test set and the remaining partition are
collectively used to train the model. In this method,
each data point is used for the same number of times
for training and once for testing. Then the error
estimates are calculated and then averaged. The errors
that were used as an indicator of the predictive
performance of the models were root mean-squared
error (RMSE), relative error (RE %), and correlation
coefficient (R). For implementation and to carry out
our experiments, the SVM and ANN models were
executed by using rapid miner software program.
3 Results and Discussion
3.1 Models Performance Comparison
The goal of this part of research is to compare the
prediction results provided by SVM based on
polynomial, Gaussian radial basis (RBF) and PUK
kernels function as well as the ANN model. The
optimization of SVM parameters is performed by
using grid search approach in the training set data
using 10-fold cross validation, and depending on
smallest RMSR error we selected the optimal
parameters of the model. The regularization constant
( the complexity parameter C ), parameters of the
- insensitive loss function, kernel parameters of
the degree d of polynomial kernel , width of RBF
kernel , and PUK kernel parameters and were
optimized. The optimal parameters of SVM based on
polynomial kernel are C=1.0, = 0.002, and d = 3.0.
The optimal parameters of SVM based on RBF kernel
are C=160.84, = 0.001, and = 0.04, and the
optimal parameters of SVM based on PUK kernel are
Table 1Comparison of the predictive performance of SVM and ANN model
Experimental data
SVM based
polynomial kernel
11.86 11.266
13.52 12.247
18.28 30.042
11.77 6.092
10.82 11.362
Error
RMSE 5.369
RE, % 22.42
R 0.934
C=160.84, = 0.02, = 85.0, and = 9.1, which give
the smallest value of RMSE.
The initial architecture of the ANN selected has
used all the eight variables in input layer with four
neurons in the hidden layer, selected by the auto
builtin function and one output neuron. The
optimization of the BP multilayer perceptron model is
done with 10-fold cross validation. After several
trials, the optimal learning rate of 0.01 and a
momentum coefficient of 0.3 are determined, and the
number of epochs is selected as 300.
After the completion of model development or
training, all the models based on SVM and ANN are
subjected to the unseen testing data set. Statistical
parameters such as root mean-squared error (RMSE),
relative error (RE %), and correlation coefficient (R)
are used to compare the predictive power of the SVM-
based and ANN based models. Results are
summarized in Table1.
It can be seen from Table 1 that the largest values
of RMSE and RE are provided by SVM based on
polynomial kernel. This indicates that the
generalization performance of SVM based on
polynomial kernel is bad. Therefore, this model does
not fit the data well. The RMSE of SVM based on
RBK kernel is bound to be lower than that of SVM
based on PUK kernel. The RE and R values provided
by both SVM models based on RBK and PUK kernels
are the same.
Therefore, the RMSE values of both SVM models
based on RBK and PUK kernels are lower than that of
ANN model. The RE and R values are found to be
close with a little difference. This comparison shows
that both SVM models based on RBK and PUK
kernels and ANN model are pretty similar in terms of
the average errors. The comparison of performance of
the nonlinear models demonstrates that the SVM
Predicted value using
SVM based SVM based ANN
RBF kernel PUK kernel
11.155 11.275 11.559
12.582 12.616 13.564
20.582 18.748 16.705
11.278 10.796 9.484
10.736 10.794 11.691
0.911 0.646 1.280
5.02 4.31 7.91
0.987 0.989 0.901
 ABAKAR & YU: SVM BASED ON PUK KERNEL IN COMPARISON TO SVM BASED ON RBF KERNEL
Table 2Anova test
Source Square DF Mean F Prob
sums squares
Between 0.003 1 0.003 0.060 0.812
Residuals 0.821 18 0.046
Total 0.824 19
based on PUK kernel has the similar ability in
predicting yarn tenacity with SVM based on RBF
kernel and ANN model.
3.2 Compare SVM Models by Significance Test
The two SVM models with RBF and PUK kernel
have been compared to know which of them is more
accurate and suitable in prediction of yarn tenacity
property, Hence, we applied two cross validation on
the training set data using SVM model based on RBF
kernel and on PUK kernel. The significance test
(T test and Anova) is performed on the resulting
performance value to decide whether the difference in
performance is significant. Table 2 shows that the
probability for random values with the same result is
0.812, these values are bigger than alpha= 0.050,
which indicates a probably difference between the
actual mean values was not significant, since
0.812 > alpha = 0.050. This means that the SVM
model based on PUK kernel has the same ability to
that of SVM model based on the RBF kernel to map
the nonlinear relation between input and output data
for predicting the yarn tenacity property. Hence, it is
possible to predict the yarn properties by using SVM
model based on PUK kernel.
4 Conclusion
It is observed that like SVM based on RBF, the
SVM model based on PUK kernel shows the same
applicability, suitability and performance to map the
nonlinear relation between input and output data for
predicting the yarn tenacity. A comparison of SVM
models based on RBF and PUK kernels with ANN
59
model shows that the two SVM models have similar
prediction performance as ANN model. The result of
this study will be helpful for further work of feature
selection to know the importance of fibre properties
on yarn tenacity by using SVM based on PUK kernel.
Acknowledgement
The authors gratefully acknowledge Shanghai
Municipal Education Commission and Shanghai
Education Development Foundation for providing
Shu Guang project (No. 10GG17).
References
1 Majumdar P K & Majumdar A, Text Res J, 74(7) (2004) 652.
2 Chattopadhyay R & Guha A, Text Prog, 35(1) (2004) 1.
3 reyen M E & Grkan P, Fibers Polym, 9(1) (2008) 87.
4 Majumdar A & Ghosh A, J Eng Fibers Fabrics, 3(4) (2008) 61.
5 Admuthe L S &. Apte S, Text Res J, 80(9) (2010) 841.
6 M. Dayik, Text Res J, 79(11), 963 (2009).
7 Yang J G, Lv Z J & Xiang Q, Yarn properties prediction
using support vector machines: an intelligent reasoning
method. Proceedings, Fourth International Conference on
Fuzzy Systems and Knowledge Discovery, Haikou, 2007, 1.
8 Ghosh A & Chatterjee P, Fibers Polym, 11(1) (2010) 84.
9 Nurwaha D & Wang X, Fibers Polym, 12(4) (2011) 546.
10 stn B, Melssen W J & Buydens L M C, Chemometrics
Intelligent Laboratory Systems, 81(1) (2006) 29.
11 Cheng Y S J & Cheng K P S, Text Res J, 74(8) (2004) 718.
12 Vapnik V N, The Nature of Statistical Learning Theory
(Springer-Verlag New York), 1995.
13 Vapnik V N, IEEE Transactions on Neural Networks,10(5)
(1999) 988.
14 Smola A J & Schlkopf B, Statistics Computing, 14(3)
(2004) 199.
15 Suykens J A K, Vandewalle J & Moor B D, Neural
Networks, 14(1) (2001) 23.
16 Mukherjee S, Osuna E & Girosi F, Nonlinear prediction of
chaotic time series using support vector machines,
Proceedings, IEEE Workshop on Neural Networks for Signal
Processing VII (Amelia Island Florida) 1997. 511.
17 Jeng J T, Chuang C C & Su S F, Fuzzy Sets Systems, 138(2)
(2003) 283.
18 Tay F E H & Cao L, Omega, 29(4) (2001) 309.
19 Chuang C C & Su S F, IEEE Transactions on Neural
Networks, 13(6) (2002) 1322.