2010 Sixth International Conference on Natural Computation (ICNC 2010)
Using Genetic Algorithms for Time Series Prediction
Cheng-Xiang Yang, Yi-Fei Zhu
School of Resources & Civil Engineering
Northeastern University
Shenyang 110004, P R China
Abstract—This paper proposes using the genetic algorithms (GAs)
for nonlinear time series prediction. A nesting evolution scheme
is designed to evolve the forecasting models. In the outer
evolution cycle, a binary-coded genetic algorithm is employed to
evolve the structures of nonlinear polynomial type models. Then
the coefficients of the evolved models are introduced and
optimized by a real-coded genetic algorithm in the inner
evolution cycle. The evolution process is repeated by using
genetic operators and the principle of ‘survival of the fittest’ until
find the satisfied results. The proposed method is applied to
deformation prediction of the dangerous rock mass in rock
engineering. The results indicate the applicability of the proposed
algorithm with enough accuracy.
Keywords-time series; genetic algorithms; nonlinear; modeling;
forecasting
I. INTRODUCTION
Decision making and planning for a variety of complex
systems involves prediction or forecasting, which is normally
carried out by investigating patterns in historical data and
speculating that the future trend will behave according to its
past pattern. Much of the historical data is recorded at specific
time intervals. This time series data provides good insight into
the behavior of the systems under study. Many efforts have
been made over the past several decades to develop and
improve time series forecasting models. Although several
different types of time series models are available [1, 2], they
have difficulties in selection of nonlinear model structures. The
approximation of linear models to complex real-world problem
is not always satisfactory. Researchers began to introduce
modern information analysis techniques for nonlinear systems,
such as artificial neural networks, grey system theory and
support vector machines, to overcome such limitations and the
results are interesting and encouraging [3-7]. This provides
potential powerful alternatives for nonlinear time series
modeling.
Recently, the evolutionary computation techniques [8] have
proved themselves robust data-based modeling tools for
complex system analysis. Based on the Darwinian theory of
natural selection, they attempt to obtain the best solution by
carrying out global optimization. They use suitable coding to
represent possible solutions for a problem and guide the search
by using genetic operators and the principle of 'survival of the
fittest'. Of these algorithms, genetic algorithms (GAs) have
established itself as a powerful search and optimization tools in
problem solving and function optimization [9, 10]. Based on
the genetic algorithms, this paper develops a nesting evolution
This work was supported by Program for New Century Excellent
Talents in University, the Special Fund for Basic Scientific Research of
Central Colleges under Grant No. N090401002, N090101001 and the SRF
for ROCS, SEM under Grant No. 20071108-4.
978-1-4244-5961-2/10/$26.00 ©2010 IEEE
algorithm for time series modeling. The motivation of the
nesting evolution method comes from the following
perspectives. First, since the complex real-world problem is
highly nonlinear, it is difficult for forecasters to choose the
exact model structure. Usually, a limit number of different
models are tried and the one with the most accurate result is
selected. However, the final selected model is not necessarily
the best for future uses due to many potential influencing
factors such as sampling variation, model uncertainty, and
structure change. By using suitable coding, the problem of
model structure selection can be eased with little extra effort
through function optimization of GAs. Second, the model
structures generated during the model identification process
may include a number of coefficients which can have great
effects on the performance of a model. As a potentially good
model with favorable structure may be removed during the
evolution process because of inappropriate parameters, a
parameter estimation procedure has to be employed to optimize
those coefficients. Then the good of fitness of each generated
model structure can be reasonably computed and assigned
before performing natural selection for further evolution. In
this study, another GA evolution cycle is used for such
parameter estimation. Thus, a nesting evolution scheme, i.e., an
outer GA evolution of model structures coupling a series of
inner GA evolution of model coefficients generated with mode
structures, is consequently designed. The proposed algorithm
is then used to model the nonlinear dynamic deformation
behavior of some dangerous rock mass to illustrate its
efficiency and accuracy.
II. GENETIC ALGORITHMS
GAs are random search algorithms based on the concepts of
natural selection, genetics and evolution. The major difference
between GAs and other classical optimization techniques is that
GAs work with a population of possible solutions, while those
classical optimization techniques work with a single solution.
Another difference is that the GAs use probabilistic transition
rules instead of deterministic rules. In a GA search process, a
group of candidate solutions, represented as genes on a
chromosome (e.g., binary strings or real numbers) in the search
space, are evolved to find better solutions through natural
selection and the genetic operators, i.e., crossover and
mutation, borrowed from natural genetics. A standard GA
consists of the following steps:
Step 1. Initialize a population of possible solutions;
Step 2. Calculate the fitness of each candidate solution in
the population;
4405
 Step 3. Select the solutions with higher fitness to take
part in evolution.
Step 4. Create new population by using GA operators on
the selected solutions.
Step 5. If the stopping criterion is satisfied, then stop the
computing and take the best solution as the final result,
otherwise, go to Step 2.
The GAs are mathematically simple yet powerful in their
search for improvement after each generation [20]. Due to
their nature, GAs have many advantages: they require no
knowledge of gradient information of the objective function;
high nonlinearities and discontinuities present on the
objective function have little effect on overall optimization
performance; they are resistant to becoming trapped in local
optima; they perform very well for large-scale optimization
problems and can be employed for a wide variety of
optimization problems.
III. GENETIC EVOLUTION SCHEME FOR NONLINEAR TIME
SERIES MODELING
A. Problem Description
Time series forecasting is an important area of forecasting
in which past observations of the same variable are collected
and analyzed to develop a model describing the underlying
relationship. It is reasonable to expect a predominant
correlation between the current observation and past
observations. Mathematically, as to observed time series {ut} (t
= 1, 2, …), the times series model can be represented as
ut = f (ut −1 , ut − 2 , , ut − p )
where p is the number of history observations and f (·) the
mapping relationship. By introducing new observations, the
model can then be used to extrapolate the time series into the
future. Namely
ut +1 = f (ut , ut −1 , , ut − p +1 )
ut + 2 = f (ut +1 , ut , , ut − p + 2 )
In (1), f (·) is usually highly nonlinear. Based on the polynomial
approximation theory, a polynomial type expression can be
used to express nonlinear relationships as has been widely used
in time series analysis. f (·) in (1) may be written as
p q
f (ut −1 , ut − 2 , , ut − p ) = ∑∑ c jk utk− i
j =1 k =1
where p, q and cjk (j=1, 2, …, p; k=1, 2, …, q) are parameters to
be determined (in some cases, some of these parameters may
be set accordingly to be zero). Of these parameters, p and q are
integer values deciding the number of input variables and the
level of the polynomial model, respectively. They construct the
frame of the model expression and can be called structure
parameters. cjk are real values deciding the model coefficients.
For example, f (·) in (1) with p =2 and q =3 can be formulated
as
2 3
ut = f (ut −1 , ut − 2 ) = ∑∑ c jk utk− j
j =1 k =1 (4)
2 3 2 3
= c11ut −1 + c u
12 t −1 +c u 13 t −1 + c21ut − 2 + c u
22 t − 2 +c u
13 t − 2
Therefore, if the structure parameters p and q and the model
coefficients cjk are determined, then the nonlinear time series
model represented in (3) is recognized.
Apparently, it is highly multimodal with large parameter
space for most real-world systems, affected by a large number
of factors with complex intercorrelations among them. The
question now becomes how we efficiently search such solution
space for a reasonable combination of those parameters that
provide overall agreement with observed time series. It is
difficult to obtain a global optimal solution using conventional
regression methods. GAs were thus implemented to find the set
of those unknown parameters that best matched the modeling
prediction with observed results.
A closer examination will easily reveal that there is a
natural nesting correlation between the model structure
parameters p and q and the model coefficients cjk. That is, the
model coefficients cjk have to be added and optimized under
given model structure, while the goodness of a model structure
decided by p and q have to be estimated with known model
coefficients. In this paper, a nesting evolution algorithm is
proposed for searching those parameters with nesting
correlation to find the global optimal result, where an outer
evolution cycle of model structures is nested by a series inner
evolution cycle of model coefficients. The details are described
in the subsequent sections.
(1) B. Outer Evolution Cycle of Model Structures
In the outer evolution cycle, model structure parameters are
generated by a standard GA procedure, starting from a initial
population of parameter sets {p, q}i (i=1, 2, …, N) (initial
generation), where N is the population size. Each parameter set
represents a model structure (i.e., the frame of the polynomial
expression without coefficients). For example, the model
structure with p=2 and q=2 can be written as
(2)
2 2
ut = f (ut −1 , ut − 2 ) = ∑∑ utk− j = ut −1 + ut2−1 + ut − 2 + ut2− 2 (5)
j =1 k =1
Since a relatively small number of parameters with integer
values are to be evolved, the binary coding method is used here
to obtain fast convergence. The chromosomes (binary strings)
of some model structures can be typically seen in Fig. 1.
(3) Subsequent generations are generated by using genetic
operators including selection, reproduction, crossover and
mutation. We can find that the algorithm can easily adjusts the
model structure through crossover and mutation on binary
strings. During crossover operation, the sub-strings (genes)
from two parent chromosomes are randomly selected to
produce a child chromosome. The mutation operation is
conducted by randomly changing the values of some bits or the
order of some sub-strings. As shown in Fig. 1.
4406
 Model structure 1 Model structure 2
(p=5, q=1) (p=2, q=2)
ut −1 + ut −2 + ut −3 + ut −4 + ut −5 ut −1 + ut2−1 + ut −2 + ut2−2
Chromosome: 01010001 Chromosome: 00100010
Crossover
New model structure 3
(p=3, q=1)
ut −1 + ut −2 + ut −3
Chromosome: 00110001
Mutation
New model structure 4
(p=2, q=4)
ut −1 + ut2−1 + ut3−1 + ut4−1 + ut −2 + ut2−2 + ut3−2 + ut4−2
Chromosome: 00100100
Figure 1. Coding and genetic operators in structure evolution cycle(A 4-bit
coding for each parameter used)
It should be noted that the fitness of each model structure
can not be estimated now because the model coefficients are
not known. Therefore, the outer evolution cycle have to be
suspended until the model coefficients have been optimized.
Having the model structures generated, necessary
coefficients are automatically introduced. These coefficients
will be optimized in the inner evolution cycle.
C. Inner Evolution Cycle of Model Coefficients
For every model structure generated in the outer evolution
cycle, the parameter set {p, q} is decoded according to (3) to
construct the mathematical expression by introducing
necessary model coefficients cjk (j=1, 2, …, p; k=1, 2, …, q).
Then another standard GA procedure is used to evolve those
coefficients in the context of the current model structure. Since
the unknown parameters (e.g., with the number of p × q) are
real values, it is more suitable and convenient to directly
represent genes as real values too. Therefore, a real-coded GA
in which all genes in a chromosome are real numbers is used.
In the present analysis, the real-coded GA with a reset
stochastic selection type of selection procedure, simulated
binary crossovers and polynomial mutations has been used. For
details of these genetic operators, we refer the reader to [11]. A
typical crossover operator in the context of parameter
estimation is shown in Fig. 2.
1
Parent chromosome 1 P1 = (c11 ,… , c11q , c21
1
,… , c12 q ,… c1jk ,
Parent chromosome 2 P2 = (c112 ,… , c12q , c21
2
,… , c22q ,… c 2jk ,
Crossover
Child chromosome 3 P3 = (c113 = χ (c11
1
, c112 ),… , c 3jk = χ (c1jk , c 2jk ), )Τ
Figure 2. Crossover operation of two candidate parameter sets in the real-
coded GA procedure (χ(•) is some crossover rule)
D. Fitness Test
In a GA evolution process for modeling problem, all
generated models have to be tested against the real-world
results to give reference values (fitness values) to control the
so-called natural selection. It is necessary to collect a set of
input-output sample cases of the studied system. For time series
modeling in the current study, these cases can be constructed
according to (1). Each case consists of a sub-series of inputs (ut-
1, ut-2, …, ut-p) selected by a moving window with size of p
from the observed series and a corresponding output ut. The
data set will be divided into two groups. One group is used as
fitness cases to obtain the optimal model. The other group is
used as testing cases to access the ability of out-of-sample
forecasting of the obtained model. One should note that the
value of p is frequently changed during the evolution process
and the data set must be reconstructed accordingly.
Fitness function that returns a measurement of the fitness of
a model is generally based on the discrepancies between the
model predictions and the observed results. It can be defined
using some statistical result of those discrepancies. In the
current study, the root of the mean square error is used, namely
1/2
⎧⎪ 1 n 2⎫
∗⎤ ⎪
Fitness = ⎨ ∑ ⎣ui − ui ⎦ ⎬
⎡ (6)
⎪⎩ n i =1
⎭⎪
where ui and ui* are the model prediction and observed result
respectively, and n the number of learning cases.
After the inner evolution cycle, the optimal set of model
coefficients for each model structure is determined and the
corresponding fitness value is assigned to the model structure
for further evolution.
IV. ALGORITHM DETAILS
The nesting evolution algorithm for time series modeling
relies on the following steps:
Step 1. Randomly generate a population of initial model
structure parameter sets {p, q}i, then step into the outer
evolution cycle;
Step 2. Estimate the fitness value of each model structure
parameter set through the following sub-steps.
Sub-step 1. Construct the learning cases and testing cases
according to (1) with the parameter p of the current model
structure;
Sub-step 2. Decode the model coefficient information (e.g.,
the number of model coefficients) of current model structure,
then step into inner evolution cycle;
Sub-step 3. Evolve the model coefficients of the current
model structure starting from an initial population of
c1pq , )Τ
coefficient sets until the termination conditions for inner
c 2pq ,)Τ evolution cycle are satisfied, where the generated coefficient
sets are tested against the fitness cases to calculate the fitness
values according to (6);
Sub-step 4. Test the current model with optimized
coefficients over the testing cases to estimate the fitness value
of the current model structure.
Step 3. If the termination conditions for outer evolution
cycle are satisfied, output the best models and terminate the
algorithm. Otherwise, go to Step 4.
Step 4. Perform genetic operations on the current
population of model structures to generate a new generation of
model structures. Then go to Step 2.
The whole evolution process is logically shown in Fig. 3.
4407
 Start Much effort has been made in the last decades to establish
different models to predict the stability state. Summarily, there
are two main kinds of methods. One kind of the methods is to
Randomly generate an initial population of develop numerical and analytical models to access the duration
model structure parameter sets {p, q} and magnitude of the movement state of the DAM. However,
affected by the complex time-dependent property of geo-
materials and many other engineering factors with considerable
Loop for each model structure
i1 = 1, N1
uncertainties, the movement of DAM is highly time-dependent
and commonly characterized with complex nonlinear dynamic
behavior. Under the conditions that the constitutive law of geo-
Construct learn cases according to the materials is far from well-known, it is very difficult to develop
current parameter value p accurate physical-based model to calculate and predict the
dynamic behavior. As an alternative, the other kind of methods
is based on data analysis of the deformation history of the
Decode the coefficient information of
current model structure
DAM that is regularly monitored during the movement. Of
these methods, deformation time series analysis has attracted
many attentions with encouraging results [3-7]. In this section,
Randomly generate an initial population of the proposed evolution modeling approach is applied for
model coefficient sets {cjk} modeling and predicting analysis of the dynamic evolution
process of a typical DAM.
Loop for each coefficient set B. Rearch Data and Implement Settings
i2 = 1, N2
To demonstrate the robustness of the proposed evolution
procedure for nonlinear time series modeling problem under
Calculate the fitness value of each consideration, we apply it to a real DAM related to the Three
coefficient set according to (6) Gorges Project in China, and the monitored deformation
history from 1978~1993 is used for modeling and predicting
Perform analysis. The data is illustrated in Fig. 4.
genetic operations
Termination No
to generate a new Before implementation of the proposed evolution procedure,
conditions population of
coefficient sets control parameters have to be pre-chosen to control the search
Yes process, and they include mainly the population size, the
Select the fitness of best coefficient set probabilities of genetic operators (crossover, and mutation), the
as the fitness of current model structure termination conditions, etc. As we have two different GA
cycles, i.e. structure evolution cycle and coefficient evolution
Perform genetic cycle respectively, two different sets of control parameters
Termination No operations to generate a have to be selected so that the search can be carried out
conditions new population of
structure parameter sets
efficiently. The implementation settings in current study are
Yes listed in Table I.
Output results
TABLE I. COMTROL PARAMETERS
Object Parameter
End Structure evolution
Population size 8
Crossover probability 0.7
Figure 3. Nesting evolution procedure for time series modeling Mutation probability 0.2
The fitness value of the best individual has
Termination criterion
It should be noted that the algorithm is based on stochastic remained unchanged for 3 generations
Parameter optimization
search. Therefore, it is a probabilistic approach in nature and
Population size 200
may arrive at different optimal solutions for different runs. To Crossover probability 0.85
eliminate the effect of this inherent variation, it is necessary to Mutation probability 0.05
perform several runs to improve the reliability of the results. Termination criterion
The fitness value of the best individual has
remained unchanged for 3 generations

V. APPLICATION TO DANGEROUS ROCK MASS

A. Problem Presentation
Stability analysis of dangerous rock mass (DAM) is a major
task in rock engineering. The collapse of DAM may lead to
serious natural hazards, each year accounting for enormous
property damage in terms of both direct and indirect costs.
C. Application Results
For the training and testing cases, several runs of
implementations described in section IV were performed with
different randomly generated initial model structures so that
each run takes a different genetic path to evolve models. After
all the runs finished, we got the optimal time series model as:

4408
 ut = 0.4359ut −1 − 0.0157ut2−1 + 0.9487ut − 2 +
0.0022ut2− 2 + 0.1872ut − 3 + 0.0016ut2− 3
Fig. 4 shows the learned and predicted deformation time
series using the evolved time series model. One can note that
the present method can attain a satisfied approximation. That
is, the evolved model found the underlying relationship
between history and future deformation series of the DAM.
The evolved model can thus be used in predicting the
movement state of the DAM and providing information for
decision makers to program prevention schedule.
25
20
Deformation/mm
15
10
Observed series
Learned series
5 Vol. 40, pp. 877-892, September 2003.
Predicted series
0 neural networks”, in the 3rd International Conference on Natural
0 6 12
Observation step
Figure 4. Learned and predicted results of the evolved nonlinear deformation
time series model (7)
VI. CONCLUSIONS
This paper presented a nesting evolution procedure for
nonlinear time series modeling. This method automatically
constructed models through an outer binary–coded GA
evolution cycle for model structure selection nested with a set
of inner real-coded GA evolution cycle for model coefficient
optimization. It can perform coupling global optimal search of
the structure as well as the coefficients of polynomial type
nonlinear model. Using this method, a typical case study on
prediction of deformation series of a real DAM related to the
Three Gorges Project is discussed. The nonlinear time series
models was evolved and examined with the monitored
(7) deformation history. The results shown that there is good
agreement between the observed and predicted deformations,
and the models obtained have interpretative forms that can be
easily used for further analysis. The present method has been
tested with encouraging success. It can be used a useful
alternative to other data-based modeling methodologies in the
prediction analysis of complex dynamic systems.
REFERENCES
[1] G.E.P. Box and G. Jenkins, “Time Series Analysis, Forecasting and
Control”, San Francisco, CA: Holden-Day, 1970.
[2] P.J. Brockwell and R.A. Davis, “Introduction to Time Series and
Forecasting”, New York, Springer, 1996.
[3] X. T. Feng, Z. Q. Zhang and P. Xu, “Adaptive and intelligent prediction
of deformation time series of high rock excavation slope”. Transactions
of Nonferrous Metals Society of China (English Edition), Vol. 9, pp.
842-846, April 1999.
[4] Z. Q. Huang, T. Jiang, Z.Q. Yue, et al, “Deformation of the central pier
of the permanent shiplock, Three Gorges Project, China: an analysis case
study”, International Journal of Rock Mechanics & Mining Sciences.
[5] C.-X. Yang and Y.-F. Zhu, “Time series analysis using GA optimized
18 Computation, Vol. IV, Los Alamitos, CA: IEEE Computer Society
Press, 2007, pp. 270-274
[6] C. Yang, X. Feng and B. Chen, “Prediction of mining induced land
subsidence using support vector machines”, in Land Subsidence-
Proceedings of the Seventh International Symposium on Land
Subsidence, Vol 2, A. G. Zhang, S. L. Gong, L. Carbognin et al., Eds.
Shanghai, Shanghai Scientific & Technical Publishers, pp.799~806,
2005.
[7] H. B. Zhao and X. T. Feng, “Study and application of genetic-support
vector machine for nonlinear displacement time series forecasting”.
Chinese Journal of Geotechnical Engineering, Vol. 25, pp. 468-471, July
2003 (in Chinese).
[8] D. E. Goldberg, “Genetic and evolutionary algorithms come of age”.
Communications of the ACM, Vol. 37, pp. 113-119, March 1994.
[9] D. E. Goldberg, “Genetic Algorithms in Search, Optimization and
Machine Learning”, MA: Addison-Wesley, Reading, 1989.
[10] J.H. Holland, “Adaptation in Natural and Artificial System”, Michigan,
The University of Michigan Press, 1975.
[11] K. Deb, “Multi-objective optimization using evolutionary algorithms”.
UK, Wiley: Chichester, 2001.
4409