Engineering Applications of Artificial Intelligence 55 (2016) 231–238

Contents lists available at ScienceDirect

Engineering Applications of Artificial Intelligence

journal homepage: www.elsevier.com/locate/engappai

A clustering-based sales forecasting scheme by using extreme learning

machine and ensembling linkage methods with applications
to computer server
Chi-Jie Lu a,n, Ling-Jing Kao b
a
Department of Industrial Management, Chien Hsin University of Science and Technology, Taiwan
b
Department of Business Management, National Taipei University of Technology, Taiwan

art ic l e i nf o a b s t r a c t

Article history: Sales forecasting has long been crucial for companies since it is important for financial planning, in-
Received 13 July 2015 ventory management, marketing, and customer service. In this study, a novel clustering-based sales
Received in revised form forecasting scheme that uses an extreme learning machine (ELM) and assembles the results of linkage
11 March 2016
methods is proposed. The proposed scheme first uses the K-means algorithm to divide the training sales
Accepted 29 June 2016
data into multiple disjointed clusters. Then, for each cluster, the ELM is applied to construct a forecasting
Available online 20 July 2016
model. Finally, a test datum is assigned to the most suitable cluster identified according to the result of
Keywords: combining five linkage methods. The constructed ELM model corresponding to the identified cluster is
Sales forecasting utilized to perform the final prediction. Two real sales datasets of computer servers collected from two
Clustering
multinational electronics companies are used to illustrate the proposed model. Empirical results showed
Ensemble learning
that the proposed clustering-based sales forecasting scheme statistically outperforms eight benchmark
Linkage method
Extreme learning machine models, and hence demonstrates that the proposed approach is an effective alternative for sales fore-
Computer server casting.
& 2016 Elsevier Ltd. All rights reserved.

1. Introduction
Sales forecasting is crucial for a company for financial planning,
inventory management, marketing, and customer service. For ex-
ample, sales forecasting has been used to estimate the required
inventory level for satisfying market demand and avoiding the
problem of over or under stocking. An effective sales forecasting
model can reduce the bullwhip effect, thereby improving a com-
pany's supply chain management and sales management efficacy
and, ultimately, increasing profits. Inaccurate sales forecasting may
cause product backlog, inventory shortages, and unsatisfied cus-
tomer demands (Luis and Richard, 2007; Thomassey, 2010; Lu
et al., 2012; Lu, 2014). Therefore, it is important to develop an
effective sales forecasting model which can generate accurate and
robust forecasting results.
A number of sales forecasting studies have been proposed in
the literature, while clustering-based forecasting models have
been adopted to improve prediction accuracy (Tay and Cao, 2001;
Prinzie and Van den Poel, 2006; Lai et al., 2009; Lu and Wang,
n works, decision trees, and autoregressive integrated moving
Correspondence to: Department of Industrial Management, Chien Hsin Uni-
versity of Science and Technology, Zhong-Li Dist., Taoyuan City 32097, Taiwan.
E-mail addresses: jerrylu@uch.edu.tw, chijie.lu@gmail.com (C.-J. Lu).
2010; Venkatesh et al., 2014; López et al., 2015). The fundamental
idea of the clustering-based forecasting model is to utilize a
clustering algorithm to partition whole training data into multiple
disjoint clusters and construct a forecasting model for every
cluster. The test data are assigned to a cluster by their similarity,
and the forecasting model of a particular cluster is used to obtain
forecasting outcomes for that cluster. Because data in the same
cluster have similar data patterns, the clustering-based forecasting
model can produce better forecasting accuracy than the forecast-
ing model built upon a complete dataset.
Even though Venkatesh et al. (2014) found that the clustering-
based approach yielded much smaller forecasting errors than the
approach of direct prediction on the entire sample without clus-
tering, the choice of the clustering approach, the similarity mea-
surement, and the predictor will impact the performance of the
clustering-based forecasting model. In the literature, a self-orga-
nizing map (SOM), the growing hierarchical self-organizing map
(GHSOM), and the K-means clustering approach were applied to
cluster data, while the support vector machine (SVM), support
vector regression (SVR), case-based reasoning (CBR), neural net-
average (ARIMA) were used as predictors in the literature (Tay and
Cao, 2001; Cao, 2003; Chang and Lai, 2005; Chang et al., 2009; Lai

http://dx.doi.org/10.1016/j.engappai.2016.06.015
0952-1976/& 2016 Elsevier Ltd. All rights reserved.
232 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238
et al., 2009; Huang and Tsai, 2009; Badge and Srivastava, 2010;
Kumar and Patel, 2010; Lu and Wang, 2010; Zhang and Yang, 2012;
Lu and Chang, 2014).
No matter which clustering approach is adopted, the linkage
method must be selected to determine the similarity between
objects so that a new observation can be assigned to the appro-
priate cluster. The single linkage, complete linkage, centroid link-
age, median linkage and Ward's linkage methods are five well-
known and frequently used linkage methods in clustering analysis,
but different linkage methods have different characteristics and
will generate different similarity measurement results (Palit and
Popovic, 2005; Hair et al., 2006; Nandi et al., 2015).
Most of the specific clustering-based forecasting models men-
tioned above use only one linkage method to calculate the similarity
of the prediction target and the clusters. However, using only one
linkage method in the clustering-based forecasting model cannot
provide a stable and effective outcome. Therefore, to solve this pro-
blem, this study proposed the use of ensemble learning to assemble
the results of different linkage methods. Ensemble learning is a
paradigm, where several intermediate classifiers or predictors are
generated and combined to finally get a single classifier or predictor.
It can be used to avoid the selection of the worst learning algorithm
and improve the performance of classification or prediction (Diet-
terich, 2000; Polikar, 2006; Yang et al., 2010; Galar et al., 2012).
Among various methods for the creation of an ensemble of classifiers,
majority voting is the most widely used ensemble technique and
considered a simple and effective scheme (Lam and Suen, 1997;
Shahzad and Lavesson, 2013). Yeon et al. (2010) also proved majority
voting is the optimal solution in the case of no concept drift. The
majority voting scheme follows democratic rules, i.e., the class with
highest number of votes is the outcome. Majority voting does not
assume prior knowledge about the problem at hand, or classifiers,
and does not require any parameter tuning once the individual
classifiers have been trained (Lam and Suen, 1997).
Instead of using conventional predictors like ARIMA or artificial
neural networks, this study used extreme learning machine (ELM)
as the predictor due to its great potential and superior perfor-
mance in practical applications (Huang et al., 2015). ELM is a novel
learning algorithm for single-hidden layer feedforward neural
networks (SLFNs), which randomly selects the input weights and
analytically determines the output weights of SLFNs (Huang et al.,
2006). Different from traditional gradient-based learning algo-
rithms for neural networks, ELM not only tends to reach the
smallest training errors but also the smallest norm of output
weights. Thus, the ELM algorithm provides much better general-
ization performance with much faster learning speed and avoids
many issues faced with the traditional algorithms, such as stop-
ping criterion, learning rate, number of epochs and local minima,
and the over tuned problems (Yeon et al., 2010). ELM has attracted
much attention in recent years and has become an important
forecasting method (Sun et al., 2008; Wong and Guo, 2010; Chen
and Ou, 2011; Lu and Shao, 2012; Wang and Han, 2015).
In this study, the clustering-based sales forecasting scheme is
implemented as follows. First, the K-means algorithm is used to
partition the whole training sales data into multiple disjoint
clusters. We adopted the K-means algorithm because it is one of
the most popular methods (Nandi et. al, 2015) and is effective and
efficient in most cases (Jain, 2010). Then, the ELM is applied to the
construct forecasting model for each cluster. Next, for a given
testing dataset, the ensemble learning based on the majority
voting scheme is utilized to combine the results of the five linkage
methods, including single linkage, complete linkage, centroid
linkage, median linkage, and Ward's linkage, to find the cluster
which the testing data set belongs to. Finally, the ELM model
corresponding to the identified cluster is used to generate the final
prediction result.
Two real, monthly aggregate sales data sets of computer ser-
vers collected from two multinational electronics companies were
utilized as an illustrative example to evaluate the performance of
the proposed model. The forecasting accuracy of the proposed
approach was compared with three single forecasting models, i.e.,
simple naïve forecast, seasonal naïve forecast, and pure ELM
models, and five clustering-based forecasting models with differ-
ent linkage methods. The model comparison shows that the pro-
posed approach provides much more accurate predictions. This
study contributes to the literature by proposing ensemble linkage
to avoid the problem caused by choosing a single linkage method
as well as by providing an application of the ELM model.
The rest of this paper is organized as follows. Section 2 gives a
brief introduction about extreme learning machine. The proposed
clustering-based sales forecasting model is thoroughly described
in Section 3. Section 4 presents the experimental results. The pa-
per is concluded in Section 5.
2. Extreme learning machine
ELM is one kind of single hidden-layer feedforward neural
networks (SLFNs). It has a three layers structure, including the
input layer, the hidden layer, and the output layer. It endeavors to
conquer the challenging issues of the traditional SLFNs such as
slow learning speed, trivial parameter tuning and poor general-
ization capability (Huang et al., 2015).
One key feature of ELM is that a researcher may randomly
choose input weights and hidden node parameters. After the input
weights and hidden nodes parameters are chosen randomly, SLFNs
become a linear system where the output weights of the network
can be analytically determined using a simple generalized inverse
operation of the hidden layer output matrices (Huang et al., 2006).
Consider N arbitrary distinct samples ( x i, yi ) where x i =
[xi1, xi2, ... , xin ]T ∈ Rn is the input data and yi = [yi1, yi2 , ... , yim ]T ∈ R m
is the target output. If the SLFNs with η hidden neurons and ac-
tivation function vector θ (x ) can approximate N samples with zero
N
error, this means ∑i = 1 ‖qi − yi ‖ = 0, where { qi , for i = 1, 2, ... , N } is
the output values of the SLFN. It can then be written compactly as:
HB = Y (1)
⎡ θ (w1⋅x1 + b1) ⋯ θ (wη⋅x1 + bη ) ⎤
⎢ ⎥
where HN × η = [θ (wi⋅xj + bi )] = ⎢ ⋮ ⋱ ⋮ ⎥
⎢⎣ θ (w1⋅xN + b1) ⋯ θ (wη⋅xN + bη )⎥⎦
N×η
represents the hidden layer output matrix of the neural network.
The ith column of H is the ith hidden node output with respect to
inputs x i . B η × m = [β1, ... , βη ] is the matrix of output weights and
βi = [βi1, βi2, ... , βim ]T is the weight vector connecting the ith hidden
node and the output nodes. wi = [wi1, wi2, ... , win ]T is the weight
vector connecting the ith hidden node and the input nodes; wi⋅xj
denotes the inner product of wi and xj . bi is the threshold (bias) of
the ith hidden node. YN × m = [y1, ... , yN ] is the matrix of targets.
Huang et al. (2006) has proven that the input weights wi and
the hidden layer biases bi of SLFNs need not be adjusted and can
be given arbitrarily. Under this assumption, the input weights wi
and hidden biases bi are randomly generated in ELM algorithm and
the output weights can be determined as simple as finding the
least-square solution to the given linear system. The minimum
norm least-square solution to the linear system (i.e. Eq.(1)) is
^ ψ
(2)
B=H Y
ψ
where H is the Moore–Penrose generalized inverse of matrix H. The
minimum norm least-square solution is unique and has the smallest
norm among all the least-square solutions (Huang et al., 2006).
 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238
In ELM, the number of hidden nodes is a critical factor for the
generalization of ELM and is only parameter needed to be de-
termined. This makes the ELM can be easily and effectively used by
avoiding tedious and time-consuming parameter tuning. However,
a shortcoming of ELM is that its output is usually unstable from
time to time because the input weights and hidden biases are
randomly chosen (Sun et al., 2008; Huang et al., 2015). To over-
come this disadvantage, for regression problem, an extended ELM
(ELME), which runs original ELM for Q times and calculates the
average values of prediction as final results, was proposed by Sun
et al. (2008).
After selecting the activation function of hidden neuron and the
neuron number of hidden layer, the ELME method generally in-
cludes four main steps. First, for the kth ELM, the input weights w ki
and hidden layer biases bik are respectively randomly initialized.
Second, under the parameters w ki and bik , the original ELM model
is utilized to obtain a prediction result qki = [q1k , q2k , ... , qNk ]. Third,
using the same input data, activation function and number of
hidden neurons, the first two steps are repeated Q times with
different random w ki and bik , k = 1, 2, ... , Q and Q different pre-
diction results qki can be obtained. Fourth, the final prediction
result q̄i is the average of all single prediction results,
1 Q
q¯ i = ∑k = 1 qki . For more detailed information, please refer to Sun
Q
et al. (2008).
3. Proposed clustering-based ensemble forecasting scheme
This study presents a clustering-based sales forecasting scheme
based on ensemble learning and ELM with five different linkage
methods. The proposed model consists of a training phase and a
testing phase. The proposed forecasting scheme is depicted in
Fig. 1. Proposed clustering-based ensemble forecasting scheme.
233
Fig. 1. The detailed description of each step in both phases is stated
in following subsection, respectively.
3.1. Training phase
The purpose of the training phase is to construct ELM for each
cluster obtained by the K-means algorithm. To achieve this goal,
four steps are taken.
Step 1: Collect and scale training data
In the first step, the training data y = ⎡⎣ y1, y2 , ... , yN ⎤⎦, which are
sales records in our empirical study, are collected and scaled by
min-max normalization method into the range of [0,1]. The for-
mula of min-max normalization is given below:
yi − min y
yi* = , ∀ i = 1, 2, ⋯N
max y − min y (3)
where yi* is the scaled data, min y is the minimum value of y and
max y is the maximum value of y. The purpose of scaling data is to
equalize the range of variables and reduce prediction errors.
Step 2: Define target variable and its predictors
The normalized training data are re-organized to define target
variables and predictors. Let yi* be the target variable and p denote
lag order. Then its corresponding predictors xi are lagged ob-
servations of yi*. That is, x i = [yi*− 1, yi*− 2 , ... , yi*− p ] = [xi1, xi2, ... , xip ].
Step 3: Cluster training data by the K-means algorithm
Let the vector of target variables be y* = [y1* , y2* , ... , yN* ] and
their predictors be X = [xij ] for i = 1, 2, ... , N and j = 1, 2, . . . , p .
234 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238

The K-means algorithm is applied to the predictors X for clustering The complete linkage (or furthest neighbor) and the single linkage
the training data into g disjoint groups. Let d denote the group, nd method have same computation procedure. However, unlike the
denote the number of training data in group d , d = 1, 2, ... , g , single linkage method, the complete linkage uses the farthest points
nd < N . The dth cluster contains the target variables in the groups to measure the similarity. In other words, the complete
y (d) = [y1(d) , y2(d) , ⋯ , yn(d) ] and their corresponding predictors linkage distance Ci(d) between the test data yi* and the cluster d is the
d
X(d) = [x ij(d) ], for i = 1, 2, ... , nd and j = 1, 2, . . . , p. maximum value of λ ik(d) , that is Ci(d) = max (λ ik(d) ). the most appropriate
k
group for the test data yi* is the cluster CG i that has the smallest Ci(d) ,
Step 4: Construct ELM model for each cluster
that is, CG i = arg min (Ci(d) ).
d
In this step, the forecasting model for each cluster is con- Unlike the farthest or nearest points in the groups to measure
structed by ELM algorithm. Let ELM(d ) present the ELM forecasting the similarity, the centroid linkage method measures the distance
model for group d . We use the ELME model as mentioned in between groups using their cluster centroid. As median are less
Section 2 to avoid the unstable result of ELM (Sun et al., 2008; impacted by outliers than mean, mean is often replaced by median
Wong and Guo, 2010). By using the sigmoidal function as activa- in distance measurement. This distance measurement using media
tion function, the ELME models with numbers of hidden nodes is known as median linkage method.
varying from 1 to 30 were constructed. For each number of hidden Let the mean of the jth predictor in group d be
1 n
nodes, an ELME model was run for 30 times and the average RMSE αj(d) = n ∑i =d1 x ij(d) = mean ({x ij(d), ∀ i = 1, 2.. , nd }) and the median of
d
of each node was calculated. The number of hidden nodes that
the jth predictor in group d be m(jd) = median ({x ij(d), ∀ i = 1, 2.. , nd }).
gives the smallest average RMSE value is selected as the best
parameter of ELME model. Then, the centroid linkage distance Ai(d) between the test data yi*
and the cluster d identified in the training phase can be computed
⎛ 2⎞
3.2. Testing phase p
by Ai(d) = ∑ j = 1 ⎜
⎝
( )
xij − a(jd) ⎟. Then, the most appropriate group
⎠
The objective of the testing phase is to assign data into a group for the test data yi* is the cluster AG i that has the smallest Ai(d), that
by ensembling the result of five linkage methods, and to obtain the is, AG i = arg min (Ai(d) ).
predicted value using the ELM of that group. To achieve this goal, d
Similar to the centroid linkage method, the median linkage
five steps are taken. ⎛ 2⎞
distance MG i of yi* can be computed by Mi(d) = ∑pj = 1 ⎜
⎝
(x ij − m(jd) ) ⎟⎠.
Step 1: Collect and scale testing dataset The most appropriate group for the test data yi* is the cluster MG i
that has the smallest Mi(d), that is, MG i = arg min (Mi(d) ).
Step 2: Define target variable and its predictors d
The first two steps of testing phase are same as those of Instead of measuring Euclidean distance, the Ward's linkage
training phase. The testing data were collected, scaled, and re-ar- method defines the distance between two clusters by within-
range to form target variables yi* and x i as described in the cluster variance of these two clusters (Palit and Popovic, 2005).
training phase. That is, y* = [y1* , y2* , ... , yn* ] is the scaled target The goal of Ward's linkage method is to minimize total within-
variable and x i is the predictors of yi*, x i = [xi1, xi2, ... , xip ]. group variance. To obtain the Ward's linkage distance W i(d) be-
tween the test data yi* and the cluster d , the first step is to include
Step 3: Apply linkage methods to appoint the testing data to a sui- the test data yi* in the cluster d and compute the centroid o(jd) of
table cluster obtained in the training phase
the new cluster d as follows
In this stage, five linkage methods (single linkage, complete
n
linkage, centroid linkage, median linkage and Ward's linkage xij + ∑k d= 1 xkj(d)
methods) are applied one by one to measure the similarity be- o(jd) = , ∀ j = 1, 2, .. , p.
1 + nd (5)
tween the testing data and clusters obtained in the training phase.
For a linkage method, the most appropriate cluster for a testing Finally, the Ward's linkage distance W i(d) is calculated by
sample is the one with minimum Euclidean distance. We use the ⎛ 2⎞
p
2
Euclidean distance to measure the similarity because it is one of Wi(d) = ∑ ⎜⎝ (x ij − o(jd) ) + nd × (a (d )
j
− o(jd) ) ⎟⎠ (6)
the most popular similarity measures in time series forecasting j=1

(Palit and Popovic, 2005).

where nd is number of data points in the group d. By Ward's
Let xkj(d) denote the kth value of jth predictor in the cluster d
linkage method, the most appropriate cluster WG i of the test data
identified in the training phase, k = 1, 2, ... , nd , j = 1, 2, . . . , p.
yi* is determined by WGi = arg min (W i(d) ).
The Euclidean distance between the predictor x i of the testing d

data yi* and the predictors xkj(d) of the training data y (d) in cluster d Step 4: Apply the majority voting to assemble the results of linkage
can be computed by method
p
⎛ 2⎞
Since different linkage methods have different characteristics,
λ ik(d) = ∑ ⎜⎝ (x ij − xkj(d) ) ⎟⎠, for i = 1, 2, ... , n, d = 1, 2, ... , g .
(4)
the testing data may be assigned to different clusters by different
j=1 linkage methods. Therefore, in step 4, we use majority voting to
Since, in the single linkage (or nearest neighbor) method, dis- combine the results of single, complete, centroid, median and
tance between two groups is defined by the Euclidean distance Ward's linkage methods. For the test data yi*, the most suitable
between the nearest points in the groups, the single linkage dis- cluster FG i is the cluster which receives the most votes from five
tance Si(d) between the test data yi* and the cluster d is the linkage methods, that is, FGi = mode (SGi, CGi, AGi, MG, WG i ).
Note that if the five assigned clusters SGi, CGi, AG i, MG and WG i are
minimum value of λ ik(d) , that is, Si(d) = min (λ ik(d) ). Then, the most
k all different or two groups receive equal votes, the most suitable
appropriate group for the test data yi* is the cluster SG i that has the cluster FG i is randomly selected from the results of five linkage
smallest Si(d) , that is, SG i = arg min (Si(d) ) methods.
d
 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238 235

Step 5: Obtain the predicted value using the ELM of the most suitable 14000
group 13000
After determining the most suitable cluster for each testing 12000
sample, the ELM forecasting model corresponding to the selected 11000

sales volume
cluster (ELM( FG i )) is obtained in the training phase and is used to 10000
generate the predicted value ( q¯ *i ) of the testing data yi*. Because
9000
the predicted value is scaled, the predicted value is re-scaled by
8000
the inverse transformation of the min-max normalization method
to obtain the final predicted value ( q̄i ) of yi*. 7000

Note that, the proposed research scheme can be used to predict 6000
more values recursively. If a recursive forecasting process is made, 5000
1 10 19 28 37 46 55 64 73 82 91 100 109 118
the first predicted value will be used as input to predict the next
Data Points
value. However, like all forecasting methods, the error will in-
crease with the proceeding of sequential prediction. Therefore, in Fig. 3. The monthly aggregate sales amount of Company B from January 2003 to
December 2012.
the proposed research scheme, the training phase only needs to be
carried out once. As long as ELM of each cluster is obtained in the
training phase, the predicted data will be incorporated into the methods, the proposed model was preferred to the EN-ELM model
cluster used to calculate it. (the proposed clustering-based forecasting scheme). The perfor-
mance of the proposed EN-ELM model was compared to those of
three single models (pure ELM, NF, and SNF) and five clustering-
4. Empirical study based forecasting models (SL-ELM, FL-ELM, CL-ELM, ML-ELM, and
WL-ELM). Pure ELM refers to the ELM model without data clus-
4.1. Data description and performance criteria
tering, NF refers to the simple naïve forecast which used the sales
volumes of previous months as predicted values, SNF refers to the
The monthly aggregate sales volume of two multinational
seasonal naïve forecast, SL-ELM refers to ELM with single linkage,
computer server companies in Taiwan is used to illustrate and
FL-ELM refers to ELM with complete linkage, CL-ELM refers to ELM
evaluate the proposed forecasting scheme of this study. Computer
with centroid linkage, ML-ELM refers to ELM with median linkage,
server sales data were used to illustrate the proposed research
and WL-ELM refers to ELM with Ward's linkage.
scheme because, compared to personal computer sales, computer
For the pure ELM, proposed EN-ELM model, and five clustering-
server sales data present relatively clear regularity, which is a
based forecasting models, five predictors as employed in Lu (2014)
desirable feature of all clustering-based forecasting methods.
However, with the rapid development in technology and innova- were used to perform one-step-ahead monthly sales forecasting.
tion, the periodicity of the computer server sales data is not as These five predictors are the previous month's sales amount (T-1),
stable as before. The cycle could be longer or shorter, and it could previous two months’ sales amount (T-2), previous three months’
contain noises. In such situations, using only the single linkage sales amount (T-3), 3-month moving average (MA3), and sales
method in a clustering-based forecasting method to determine the amount at the same time last year (T-12). These predictors are
most suitable cluster cannot produce a reliable outcome. In this selected according to Lu (2014).
study, the use of computer sales data was preferred as a means to The prediction performance is evaluated using the following
demonstrate that the solution of the ensemble learning provided performance measures: the root mean square error (RMSE), mean
in this study can overcome this problem. absolute error (MAE), mean absolute percentage error (MAPE),
The computer server sales data applied in this study were col- root mean square percentage error (RMSPE), and normalized mean
lected from January 2003 to December 2012, and their time series square error (NMSE). The lower values of RMSE, MAD, MAPE,
plots are presented in Figs. 2 and 3. In total, there are 120 data points RMSPE, and NMSE indicate that the predicted result is closer to the
in the data set. The data set was divided into training and testing data actual value. The definitions of these criteria are given as follows:
sets. The first 96 data points (80% of the total sample points) are used n
as the training samples, while the remaining 24 data points (20% of 1
RMSE = ∑ (yi − qi )2
the total sample points) are employed as the testing sample for n i=1 (7)
measuring out-of-sample forecasting ability.
Because the proposed forecasting method consists of the
K-means algorithm, ensemble learning and ELM with five linkage 1
n
MAE = ∑ yi − qi
n i=1 (8)
4000

3500
n
1 yi − qi
3000 MAPE = ∑
sales volume

n i=1
yi (9)
2500

2000
n ⎛
y −q⎞
2
1
1500
RMSPE = ∑ ⎜ i i⎟
n i = 1 ⎝ yi ⎠ (10)
1000
1 10 19 28 37 46 55 64 73 82 91 100 109 118
Data Point n
∑i = 1 ( yi − qi )2
Fig. 2. The monthly aggregate sales volume of Company A from January 2003 to
NMSE = n
∑i = 1 ( yi − y )2 (11)
December 2012.
236 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238

0.210 Table 1
Sensitivity analysis of Company A's clustering-based forecasting models.
0.180
Model Number of MAPE RMSPE RMSE MAE NMSE
0.150
clusters
0.120
RMSE

SL-ELM 2 3.59% 4.81% 122.48 93.72 0.111

0.090 3 3.42% 4.53% 114.73 87.39 0.094
4 6.14% 8.24% 213.21 158.09 0.364
0.060 5 8.27% 10.72% 263.69 203.76 0.556

0.030 FL-ELM 2 3.72% 4.35% 115.94 97.51 0.101

3 2.84% 3.59% 98.46 74.69 0.069
0.000 4 5.04% 7.56% 207.45 128.82 0.339
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 5 6.51% 9.17% 295.97 187.97 0.682
Number of hidden nodes
CL-ELM 2 3.51% 4.11% 104.37 88.60 0.086
Fig. 4. Sensitivity analysis for the pure ELM model of Company A. 3 3.62% 4.14% 105.83 89.80 0.080
4 6.53% 11.29% 264.01 156.47 0.557
where yi and qi represent the actual and predicted value at month 5 8.47% 10.96% 271.30 205.38 0.634

i, respectively; n is the total number of testing data points; ȳ is the

ML-ELM 2 3.31% 4.21% 101.37 81.12 0.084
average of yi . 3 3.82% 3.96% 98.39 95.46 0.067
4 3.44% 3.95% 96.56 86.91 0.063
5 5.43% 6.41% 194.37 152.55 0.286
4.2. Empirical results
WL-ELM 2 3.71% 4.18% 106.89 96.11 0.088
3 2.93% 3.54% 92.35 75.76 0.061
To construct the pure ELM model, a sensitivity study was 4 3.07% 4.10% 103.29 78.06 0.081
conducted to determine the optimal number of hidden nodes, 5 6.73% 9.06% 231.33 171.69 0.440
which gives the smallest average RMSE value. For example, in
Proposed En- 2 3.05% 3.98% 96.73 77.52 0.082
Fig. 4, the pure ELM model with 25 hidden nodes is the optimal for ELM 3 2.09% 2.69% 66.04 52.25 0.031
Company A's sales data because it has the smallest average RMSE 4 2.90% 4.46% 111.43 73.15 0.112
values. Because ELM was also used to construct the forecasting 5 5.08% 7.25% 198.88 134.83 0.352
model for the proposed EN-ELM model and five alternatives (SL-
ELM, FL-ELM, CL-ELM, ML-ELM, and WL-ELM), the sensitivity
analysis was applied as described in the pure ELM model to de- Table 2
Summary of Company A's forecasting results by the proposed En-ELM model and
termine the optimal hidden node numbers for each of them. eight competing methods.
In addition to the hidden node numbers, the cluster number of
the clustering-based forecasting model is the other factor influen- Methods MAPE RMSPE RMSE MAD NMSE
cing the forecasting result because either too many or too few
SNF 4.00% 5.53% 138.24 100.96 0.136
clusters can increase testing error. To obtain the better forecasting NF 15.06% 17.71% 446.27 379.38 1.415
result, a sensitivity analysis with 2 to 5 clusters was conducted for Pure ELM 3.94% 5.49% 133.37 96.34 0.126
the proposed EN-ELM model and five alternatives (SL-ELM, FL-ELM, SL-ELM(3) 3.42% 4.53% 114.73 87.39 0.094
FL-ELM(3) 2.84% 3.59% 98.46 74.69 0.069
CL-ELM, ML-ELM, and WL-ELM). The cluster number with the CL-ELM(2) 3.51% 4.11% 104.37 88.60 0.086
minimum testing error is the optimal cluster number for the model. ML-ELM(2) 3.31% 4.21% 101.37 81.12 0.084
Table 1 depicts the forecasting results of all clustering-based WL-ELM(3) 2.93% 3.54% 92.35 75.76 0.061
En-ELM(3) 2.09% 2.69% 66.04 52.25 0.031
models with different cluster numbers. By comparing the values of
MAPE, RMSPE, RMSE, MAE, and NMSE, it was found that the best
number of clusters for SL-ELM, FL-ELM, WL-ELM, and the proposed
EN-ELM is 3, and the best number of clusters for CL-ELM and ML- Table 3
ELM is 2. For better presentation, parentheses are used to indicate Summary of Company B's forecasting results by the proposed En-ELM model and
eight competing methods.
the best cluster number for each clustering-based forecasting
model. For example, EN-ELM(3) refers to EN-ELM with 3 clusters. Methods MAPE RMSPE RMSE MAE NMSE
The forecasting results of Company A's sales data are summar-
SNF 4.36% 6.66% 601.52 398.71 0.191
ized in Table 2. As presented in Table 2, compared to its competing
NF 8.60% 12.10% 1176.59 840.96 0.729
models, the proposed EN-ELM model has the lowest MAPE, RMSPE, Pure ELM 3.95% 5.04% 485.08 374.11 0.124
RMSE, MAD, and NMSE values. This result indicates that the pro- SL-ELM(4) 3.98% 5.01% 476.38 373.43 0.120
FL-ELM(3) 3.71% 4.74% 456.69 351.56 0.110
posed EN-ELM has the lowest number of prediction errors.
CL-ELM(4) 3.79% 4.95% 497.44 368.10 0.130
Because Company A's and Company B's sales data were ana- ML-ELM(3) 3.31% 4.04% 397.47 318.52 0.083
lyzed by the same procedure, only the forecasting result of Com- WL-ELM(3) 3.57% 4.78% 464.39 339.73 0.114
pany B is reported (Table 3) to save space. As both Tables 2 and 3 En-ELM(3) 2.22% 2.58% 249.72 210.88 0.033

reveal, the proposed EN-ELM has the lowest number of prediction

errors and outperforms competing alternatives. And, compared to
SNF, NF, and pure ELM, all clustering-based models have better
prediction accuracy. This finding is consistent with previous stu-
dies (Lu and Wang, 2010; Venkatesh et al., 2014; López et al., 2015)
which stated that the clustering-based forecasting models can be
used to improve prediction performance.
4.3. Significance test
To further evaluate whether the difference in Tables 2 and 3
between the proposed EN-ELM model and competing models is
statistically significant, the Wilcoxon signed-rank test, a popular
 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238 237

Table 4
Wilcoxon signed-rank test resultsa between the En-ELM and eight competing models.

Models NF SNF Pure ELM SL-ELM FL-ELM CL-ELM ML-ELM WL-ELM

Company A
En-ELM 2.086 (0.037)** 4.086 (0.000)** 2.084 (0.040)** 1.829 (0.067)* 1.657 (0.097)* 1.857 (0.053)* 1.743 (0.081)* 1.686 (0.092)*

Company B
En-ELM 3.086 (0.002)** 3.657 (0.000)** 2.529 (0.011)** 2.472 (0.013)** 2.114 (0.034)** 2.220 (0.028)** 1.714 (0.086)* 2.000 (0.048)**

a
p-value in parentheses.
*
p-valueo 0.10.
**
p-valueo 0.05.

nonparametric technique, was employed. Please refer to Diebold Table 5

and Mariano (1995) and Pollock et al. (2005) for the details of the Model comparison between the En-SVR model and competing models with dif-
ferent clustering method and predictors.
Wilcoxon signed-rank test.
Table 4 reports the Z statistic and p-values of the two-tailed Comparison group Methods Company A Company B
Wilcoxon signed-rank test for the squared errors between the
proposed EN-ELM model and its competing models. It shows that MAPE RMSPE MAPE RMSPE
the forecasting error of the EN-ELM model is significantly lower
1 Pure SVR 3.92% 5.45% 4.05% 5.07%
than those of its competing models. Therefore, it can be concluded SL-SVR 3.35% 4.49% 3.96% 5.11%
that the proposed EN-ELM model significantly outperforms the FL-SVR 2.85% 3.54% 3.77% 4.65%
alternatives in sales forecasting. CL-SVR 3.47% 4.08% 3.78% 4.86%
ML-SVR 3.36% 4.18% 3.34% 4.07%
WL-SVR 2.86% 3.56% 3.53% 4.83%
4.4. Effectiveness evaluation of ensemble learning in forecasting
En-SVR 2.11% 2.73% 2.27% 2.65%

To further validate the effectiveness of the proposed clustering- 2 Pure ELM 3.94% 5.49% 3.95% 5.04%
based forecasting scheme with ensemble learning, the K-means P-SL-ELM 3.59% 4.60% 3.69% 4.90%
P-FL-ELM 2.91% 3.57% 3.73% 4.75%
algorithm is replaced with partitioning around medoids (PAM)
P-CL-ELM 3.61% 3.99% 3.71% 4.73%
(Kaufman and Rousseeuw, 1990), ELM is replaced with SVR P-ML-ELM 3.52% 4.02% 3.47% 4.11%
(Vapnik, 1999) in the proposed forecasting scheme (Fig. 1), and the P-WL-ELM 2.78% 3.58% 3.39% 4.63%
prediction accuracy among the proposed EN-ELM, EN-SVR (the P-En-ELM 2.20% 2.86% 2.32% 2.78%
K-means algorithm, ensemble learning, and SVR), P-EN-ELM (PAM,
3 Pure SVR 3.92% 5.45% 4.05% 5.07%
ensemble learning, and ELM), and P-EN-SVR (PAM, ensemble
P-SL-SVR 3.64% 4.65% 3.93% 5.07%
learning, and SVR) is compared. Even though the model with en- P-FL-SVR 3.06% 3.87% 3.83% 4.99%
semble learning (EN-ELM) has proven to have better performance P-CL-SVR 3.44% 3.95% 3.76% 4.64%
than models with single linkage shown in Section 4.3, the pre- P-ML-SVR 3.48% 4.08% 3.38% 4.05%
P-WL-SVR 3.11% 3.51% 3.84% 4.73%
diction accuracy between the new models (EN-SVR, P-EN-ELM and
P-En-SVR 2.21% 2.89% 2.35% 2.81%
P-EN-SVR) and those with five single linkage methods for com- Proposed model En-ELM 2.09% 2.69% 2.22% 2.58%
pleteness was still compared.
PAM was chosen because PAM is not only a well-known
squared error-based clustering algorithm but also one of the most the advantage of PAM and to make the data in clusters relatively
popular k-medoids clustering algorithms. Compared to the homogeneous, which leads to the better forecasting performance
K-means algorithm, PAM is more robust to noise and outliers (Xu of ELM. This finding is also consistent with Jain (2010), who shows
and Wunsch, 2005). SVR is chosen because it is an effective ma- that the K-means algorithm is effective and efficient in most cases,
chine-learning algorithm for solving nonlinear regression estima- and with Huang et al. (2006), who shows that ELM and SVR have a
tion problems and has been successfully used in sales forecasting similar prediction performance.
(Lu and Wang, 2010; Lu et al., 2012; Lu, 2014). The parameters of
SVR were determined according to Lu (2014), while all linkage
methods were applied as illustrated in Section 4.2. 5. Conclusion
The results of model comparison are reported in Table 5. The
first comparison group consists of the models using SVR as a Business decisions, such as production, inventory management,
forecaster, the second comparison group consists of the models financial planning, and sales, relies highly on sales forecasting.
using PAM as the clustering method, and the third comparison Enhancing forecasting accuracy can improve a firm's decision-
group consists of the models using PAM as the clustering method making efficacy and lower operational costs. This is particularly
and using SVR as a forecaster. As shown in Table 5, regardless of true for premium products, such as computer servers, which have
the clustering methods and forecasters, the models using en- high production costs.
semble learning (EN-SVR, P-EN-ELM, and P-EN-SVR) always have To enhance forecasting accuracy, this study proposes a novel
better forecasting accuracy than those models in the same com- clustering-based sales forecasting scheme which uses the K-means
parison group. algorithm for data clustering, ensemble learning with majority
Moreover, the proposed model EN-ELM has the lowest MAPE voting for integrating the results of the five linkage methods (i.e.,
and RMSPE among all the models using ensemble learning. The single linkage, complete linkage, centroid linkage, median linkage,
reason why P-EN-ELM and P-EN-SVR do not have better fore- and Ward's linkage methods), and extreme learning machine
casting accuracy might be due to the fact that computer server (ELM) as a predictor. Two- month aggregate sales data of two
sales data are more regular and have fewer noises and outliers. computer server companies in Taiwan are used to evaluate the
This characteristic of computer server sales data fails to highlight performance of the proposed method.
238 C.-J. Lu, L.-J. Kao / Engineering Applications of Artificial Intelligence 55 (2016) 231–238
Empirical results show that the proposed sales forecasting
scheme produces the best forecasting results and statistically
outperforms models without clustering (NF, SNF, and pure ELM)
and models with the single linkage method (such as SL-ELM, FL-
ELM, CL-ELM, ML-ELM, and WL-ELM). Replacing the K-means al-
gorithm with PAM and ELM with SVR in the proposed forecasting
scheme will not diminish the forecasting accuracy. Therefore, this
study validates the concept that clustering-based forecasting
models with ensemble learning can be used to improve the pre-
diction performance of a single forecasting model. Since the sales
data adopted in this study are relatively simple and limited in
number, future research should investigate the performance of
proposed research schemes in more sophisticated cases.
Acknowledgments
This work is partially supported by the Ministry of Science and
Technology, Taiwan, R.O.C. under Grant no. MOST 103-2221-E-
231-003-MY2. The authors also gratefully acknowledge the helpful
comments and suggestions of the reviewers, which have improved
the presentation.
References
Badge, J., Srivastava, N., 2010. Selection and forecasting of stock market patterns
using K-means clustering. Int. J. Stat. Syst. 5, 23–27.
Cao, L.J., 2003. Support vector machines experts for time series forecasting. Neu-
rocomputing 51, 321–339.
Chang, P.C., Lai, C.Y., 2005. A hybrid system combining self-organizing maps with
case-based reasoning in wholesaler's new-release book forecasting. Expert Syst.
Appl. 29 (1), 183–192.
Chang, P.C., Liu, C.H., Fan, C.F., 2009. Data clustering and fuzzy neural network for
sales forecasting: a case study in printed circuit board industry. Knowl.-Based
Syst. 22 (5), 344–355.
Chen, F.L., Ou, T.Y., 2011. Sales forecasting system based on Gray extreme learning
machine with Taguchi method in retail industry. Expert Syst. Appl. 38 (3),
1336–1345.
Diebold, F.X., Mariano, R.S., 1995. Comparing predictive accuracy. J. Bus. Econ. Stat.
13, 253–263.
Dietterich, T.G., 2000. Ensemble methods in machine learning. Lect. Notes Comput.
Sci. 1857, 1–15.
Galar, M., Fernández, A., Barrenechea, E., Bustince, H., Herrera, F., 2012. A review on
ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-
based approaches. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 42 (4),
463–484.
Hair, J.F., Black, W.C., Babin, B., Anderson, R.E., Tatham, R.L., 2006. Multivariate Data
Analysis, 6th ed. Prentice-Hall, New York.
Huang, C.L., Tsai, C.Y., 2009. A hybrid SOFM-SVR with a filter-based feature selection
for stock market forecasting. Expert Syst. Appl. 36 (2), 1529–1539.
Huang, G., Huang, G.B., Song, S., You, K., 2015. Trends in extreme learning machines:
a review. Neural Netw. 61, 32–48.
Huang, G.B., Zhu, Q.Y., Siew, C.K., 2006. Extreme learning machine: theory and
applications. Neurocomputing 70 (1), 489–501.
Jain, A.K., 2010. Data clustering: 50 years beyond K-means. Pattern Recognit. Lett.
31 (8), 651–666.
Kaufman, L., Rousseeuw, P., 1990. Finding Groups in Data: An Introduction to
Cluster Analysis. John Wiley & Sons, New Jersey.
Kumar, M., Patel, N.R., 2010. Using clustering to improve sales forecasts in retail
merchandising. Ann. Oper. Res. 174 (1), 33–46.
Lai, R.K., Fan, C.Y., Huang, W.H., Chang, P.C., 2009. Evolving and clustering fuzzy
decision tree for financial time series data forecasting. Expert Syst. Appl. 36,
3761–3773.
Lam, L., Suen, S.Y., 1997. Application of majority voting to pattern recognition: an
analysis of its behavior and performance. IEEE Trans. Syst. Man Cybern. Part A:
Syst. Humans 27 (5), 553–568.
López, K.L., Gagné, C., Castellanos-Dominguez, G., Orozco-Alzate, M., 2015. Training
subset selection in hourly Ontario energy price forecasting using time series
clustering-based stratification. Neurocomputing 156, 268–279.
Lu, C.J., 2014. Sales forecasting of computer products based on variable selection
scheme and support vector regression. Neurocomputing 128, 491–499.
Lu, C.J., Wang, Y.W., 2010. Combining independent component analysis and grow-
ing hierarchical self-organizing maps with support vector regression in product
demand forecasting. Int. J. Prod. Econ. 128 (2), 603–613.
Lu, C.J., Shao, Y.E., 2012. Forecasting computer products sales by integrating en-
semble empirical mode decomposition and extreme learning machine. Math.
Probl. Eng. 2012, 2012. http://dx.doi.org/10.1155/2012/831201 2012.
Lu, C.J., Chang, C.C., 2014. A hybrid sales forecasting scheme by combining in-
dependent component analysis with K-meanss clustering and support vector
regression. Sci. World J. 2014, 2014. http://dx.doi.org/10.1155/2014/624017.
Lu, C.J., Lee, T.S., Lian, C.M., 2012. Sales forecasting for computer wholesalers: a
comparison of multivariate adaptive regression splines and artificial neural
networks. Decis. Support Syst. 54 (1), 584–596.
Luis, A., Richard, W., 2007. Improved supply chain management based on hybrid
demand forecasts. Appl. Soft Comput. 7 (1), 136–144.
Nandi, A.K., Fa, R., Abu-Jamous, B., 2015. Integrative Cluster Analysis in Bioinfor-
matics. John Wiley & Sons, West Sussex, UK.
Palit, A.K., Popovic, D., 2005. Computational Intelligence in Time Series Forecasting:
Theory and Engineering Applications (Advances in Industrial Control). Spring-
er-Verlag Inc., New Jersey, New York.
Polikar, R., 2006. Ensemble based systems in decision making. IEEE Circuits Syst.
Mag. 6 (3), 21–45.
Pollock, A.C., Macaulay, A., Thomson, M.E., Onkal, D., 2005. Performance evaluation
of judgemental directional exchange rate predictions. Int. J. Forecast. 21,
473–489.
Prinzie, A., Van den Poel, D., 2006. Incorporating sequential information into tra-
ditional classification models by using an element/position-sensitive SAM.
Decis. Support Syst. 42 (2), 508–526.
Shahzad, R.K., Lavesson, N., 2013. Comparative analysis of voting schemes for en-
semble-based malware detection. J. Wirel. Mob. Netw. Ubiquitous Comput.
Dependable Appl. 4 (1), 98–117.
Sun, Z.L., Choi, T.M., Au, K.F., Yu, Y., 2008. Sales forecasting using extreme learning
machine with applications in fashion retailing. Decis. Support Syst. 46 (1),
411–419.
Tay, F.E.H., Cao, L.J., 2001. Improved financial time series forecasting by combining
support vector machines with self-organizing feature map. Intell. Data Anal. 5
(4), 339–354.
Thomassey, S., 2010. Sales forecasts in clothing industry: the key success factor of
the supply chain management. Int. J. Prod. Econ. 128 (2), 470–483.
Vapnik, V.N., 1999. An overview of statistical learning theory. IEEE Trans. Neural
Netw. 10, 988–999.
Venkatesh, K., Ravi, V., Prinzie, A., Van den Poel, D., 2014. Cash demand forecasting
in ATMs by clustering and neural networks. Eur. J. Oper. Res. 232 (2), 383–392.
Wang, X., Han, M., 2015. Improved extreme learning machine for multivariate time
series online sequential prediction. Eng. Appl. Artif. Intell. 40, 28–36.
Wong, W.K., Guo, Z.X., 2010. A hybrid intelligent model for medium-term sales
forecasting in fashion retail supply chains using extreme learning machine and
harmony search algorithm. Int. J. Prod. Econ. 128 (2), 614–624.
Xu, R., Wunsch, D., 2005. Survey of clustering algorithms. IEEE Trans. Neural Netw.
16 (3), 645–678.
Yang, P., Yang, Y.H., Zhou, B.B., Zomaya, A.Y., 2010. A review of ensemble methods in
bioinformatics. Curr. Bioinform. 5 (4), 296–308.
Yeon, K., Song, M.S., Kim, Y., Choi, H., Park, C., 2010. Model averaging via penalized
regression for tracking concept drift. J. Comput. Graph. Stat. 19 (2), 457–473.
Zhang, J., Yang, Y., 2012. BP neural network model based on the K-meanss clustering
to predict the share price. In: Proceedings of the IEEE 2012 Fifth International
Joint Conference on Computational Sciences and Optimization, pp. 181–184.