Least Squares Support Vector Machines Classifiers :a multi two-spiral benchmark problem
Lukas, J.A.K Suykens, J. Vandewalle
Dept. of Electrical Engineering ESAT-SISTAKatholieke Universiteit LeuvenKasteelpark Arenberg 10, B-3001 Leuven, Belgium.email : firstname.lastname@example.org
keywords : Support Vector Machines, least squares SVM, pattern recognition, classifier.
Classification problems have arisen in many applications, attracting many researches to develop advanced classifier techniques.A method called Support Vector Machines (SVM) for pattern recognition and function estimation has been introduced by Vapnik (1995) in the framework of statistical learning theory. Since then there is a growing interest on this kernel method for itsinteresting features. In this paper, a least squares version (LS-SVM) is explained. LS-SVM expresses the training in terms of solving a set of linear equations instead of quadratic programming as for the standard SVM case. Iterative training algorithm for LS-SVM based on a conjugate gradient method is then applied. The LS-SVM is then able to solve large scale classification problems which is illustrated on a multi two-spiral benchmark problem.
Support Vector Machines (SVM) for solving pattern recognition and nonlinear function estimation problems have been introduced in . The idea of SVM is mapping the training data nonlinearly into a higher-dimensional featurespace, then construct a separating hyperplane with maximum margin there. This yields a nonlinear decision boundary in input space. By the use of a kernel function, either polynomial, splines, radial basis function (RBF) or multilayer perceptron, it is possible to compute the separating hyperplane without explicitly carrying out the mapinto the feature space. While classical Neural Networks techniques suffer from the existence of many local minima,SVM solutions are obtained from quadratic programming problems possessing a global solution.Recently, least squares (LS) versions of SVM have been investigated for classification  and function estimation. In these LS-SVM formulations one computes the solution by solving a linear system instead of quadratic programming. This is due to the use of equality instead of inequality constraints in the problem formulation. In [1, 4]such linear systems have been called Karush-Kuhn-Tucker (KKT) systems and their numerical stability has beeninvestigated. This linear system can be efficiently solved by iterative methods such as conjugate gradient , andenables solving large scale classification problems. As an example we show the excellent performance on a multitwo-spiral benchmark problem, which is known to be a difficult test case for neural network classifiers .
LEAST SQUARES SUPPORT VECTOR MACHINES
Given a training set of N data points
N k k k
-th input pattern and
-thoutput pattern, the classifier can be constructed using the support vector method in the form
N k k k k
b x x K y sign x y
are called support values and
is a constant. The
is the kernel, which can be either
x x x x K
T k k
d T k k
x x x x K
(polynomial SVM of degree d);