Stable Mutations for Evolutionary

Algorithms Andrzej Obuchowicz

Visit to download the full and correct content document:
https://textbookfull.com/product/stable-mutations-for-evolutionary-algorithms-andrzej-
obuchowicz/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Applied Evolutionary Algorithms in Java Ghanea Hercock

Robert

https://textbookfull.com/product/applied-evolutionary-algorithms-
in-java-ghanea-hercock-robert/

Metaheuristic and Evolutionary Computation: Algorithms

and Applications Hasmat Malik

https://textbookfull.com/product/metaheuristic-and-evolutionary-
computation-algorithms-and-applications-hasmat-malik/

Optimization Using Evolutionary Algorithms and

Metaheuristics: Applications in Engineering 1st Edition
Kaushik Kumar

https://textbookfull.com/product/optimization-using-evolutionary-
algorithms-and-metaheuristics-applications-in-engineering-1st-
edition-kaushik-kumar/

Stable isotope forensics : methods and forensic

applications of stable isotope analysis Second Edition
Meier-Augenstein

https://textbookfull.com/product/stable-isotope-forensics-
methods-and-forensic-applications-of-stable-isotope-analysis-
second-edition-meier-augenstein/
Artificial Intelligence in Value Creation Andrzej
Wodecki

https://textbookfull.com/product/artificial-intelligence-in-
value-creation-andrzej-wodecki/

Phage Therapy A Practical Approach Andrzej Górski

https://textbookfull.com/product/phage-therapy-a-practical-
approach-andrzej-gorski/

The Grodynski Brigade 1st Edition Grodynski Andrzej

https://textbookfull.com/product/the-grodynski-brigade-1st-
edition-grodynski-andrzej/

The English Countryside: Representations, Identities,

Mutations 1st Edition David Haigron (Eds.)

https://textbookfull.com/product/the-english-countryside-
representations-identities-mutations-1st-edition-david-haigron-
eds/

Aneurysms-Osteoarthritis Syndrome. SMAD3 Gene Mutations

1st Edition Denise Van Der Linde

https://textbookfull.com/product/aneurysms-osteoarthritis-
syndrome-smad3-gene-mutations-1st-edition-denise-van-der-linde/
Studies in Computational Intelligence 797

Andrzej Obuchowicz

Stable
Mutations for
Evolutionary
Algorithms
Studies in Computational Intelligence

Volume 797

Series editor
Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland
e-mail: kacprzyk@ibspan.waw.pl
The series “Studies in Computational Intelligence” (SCI) publishes new develop-
ments and advances in the various areas of computational intelligence—quickly and
with a high quality. The intent is to cover the theory, applications, and design
methods of computational intelligence, as embedded in the fields of engineering,
computer science, physics and life sciences, as well as the methodologies behind
them. The series contains monographs, lecture notes and edited volumes in
computational intelligence spanning the areas of neural networks, connectionist
systems, genetic algorithms, evolutionary computation, artificial intelligence,
cellular automata, self-organizing systems, soft computing, fuzzy systems, and
hybrid intelligent systems. Of particular value to both the contributors and the
readership are the short publication timeframe and the world-wide distribution,
which enable both wide and rapid dissemination of research output.

More information about this series at http://www.springer.com/series/7092

Andrzej Obuchowicz

Stable Mutations
for Evolutionary Algorithms

123
Andrzej Obuchowicz
Faculty of Computer Science, Electrical
and Control Engineering
Institute of Control and Computation
Engineering
University of Zielona Góra
Zielona Góra, Poland

ISSN 1860-949X ISSN 1860-9503 (electronic)

Studies in Computational Intelligence
ISBN 978-3-030-01547-3 ISBN 978-3-030-01548-0 (eBook)
https://doi.org/10.1007/978-3-030-01548-0

Library of Congress Control Number: 2018956278

© Springer Nature Switzerland AG 2019

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, express or implied, with respect to the material contained herein or
for any errors or omissions that may have been made. The publisher remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
To my wife
Beata
and our children
Maria, Adam, and Tomasz
Contents

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2 Foundations of Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . 9
2.1 Basic Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Standard Evolutionary Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2.2 Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2.3 Evolutionary Programming . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.4 Evolutionary Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.5 Evolutionary Search with Soft Selection . . . . . . . . . . . . . . 19
2.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3 Stable Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1 Stable Random Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.1 Definition of Stable Distributions . . . . . . . . . . . . . . . . . . . 23
3.1.2 Chosen Properties of Stable Distributions . . . . . . . . . . . . . 26
3.1.3 Simulation of a-Stable Random Variables . . . . . . . . . . . . . 28
3.1.4 Stable Random Variable and the Evolutionary
Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.2 Stable Random Vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2.1 Definition of the Stable Random Vector . . . . . . . . . . . . . . 33
3.2.2 Selected Properties of the Stable Random Vector . . . . . . . 34
3.2.3 Non-isotropic Stable Random Vector . . . . . . . . . . . . . . . . 36
3.2.4 Isotropic Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.2.5 Isotropic Distribution Based on an a-Stable Generator . . . . 41
3.2.6 Isotropic Stable Random Vector . . . . . . . . . . . . . . . . . . . . 43
3.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

vii
viii Contents

4 Non-isotropic Stable Mutation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

4.1 Dead Surrounding Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
4.1.1 Dead Surrounding Versus Extremum Localization . . . . . . . 51
4.1.2 Saddle Crossing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1.3 Optimization of Multimodal Functions . . . . . . . . . . . . . . . 60
4.2 Symmetry Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2.1 Local Convergence of ð1 þ 1ÞESN;a . . . . . . . . . . . . . . . . . 66
4.2.2 Saddle Crossing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.2.3 Symmetry Effect Versus Global Extremum Searching . . . . 71
4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5 Isotropic Stable Mutation . . . . . . . . . . . . . . . . . . . . . . . . . ........ 77
5.1 Dead Surrounding Effect . . . . . . . . . . . . . . . . . . . . . . . ........ 77
5.1.1 Probability Density Function of the Norm
of the Isotropic Stable Vector . . . . . . . . . . . . . . . . . . . . . . 77
5.1.2 Soft Selection Versus the Dead Surrounding . . . . . . . . . . . 80
5.2 Local Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.2.1 Progress Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
5.2.2 Convergence Analysis for n ¼ 3 . . . . . . . . . . . . . . . . . . . . 83
5.3 Exploration Versus Exploitation . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.3.1 Local Convergence of the ð1 þ 1ÞESI;a Strategy . . . . . . . . 86
5.3.2 Saddle Crossing: The ð1; 2ÞESI;a Strategy . . . . . . . . . . . . . 89
5.3.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.4 Robustness Analysis of Stable Mutation . . . . . . . . . . . . . . . . . . . . 91
5.4.1 General Searching Space . . . . . . . . . . . . . . . . . . . . . . . . . 92
5.4.2 Isotropic Stable Mutation Effectiveness in F . . . . . . . . . . 93
5.4.3 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.5 Adaptation of Stable Mutation Parameters . . . . . . . . . . . . . . . . . . 97
5.5.1 Adaptation Strategy for ð1 þ 1ÞESI;a . . . . . . . . . . . . . . . . . 97
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6 Stable Mutation with the Discrete Spectral Measure . . . . . . . . . . . . 107
6.1 Stable Distributions with the DSM . . . . . . . . . . . . . . . . . . . . . . . 107
6.2 Selection of the Optimal Stable Distribution . . . . . . . . . . . . . . . . . 110
6.3 DSM Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6.3.1 DSM Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
6.3.2 Simulation Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
6.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
Contents ix

7 Isotropic Mutation Based on an a-Stable Generator . . . . . . . . . . . . . 123

7.1 Dead Surrounding Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
7.2 Local Convergence Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.2.1 Local Convergence of the ð1 þ 1ÞESS;a Strategies . . . . . . . 125
7.2.2 Local Convergence of the ð1 þ kÞESS;a Strategies . . . . . . . 126
7.3 Exploitation and Exploration: ESTSS;a . . . . . . . . . . . . . . . . . . . . . 128
7.3.1 Precision of Local Extremum Localization . . . . . . . . . . . . 128
7.3.2 Saddle Crossing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
7.3.3 Exploitation Contra Exploration: The Pareto Front . . . . . . . 131
7.4 Global Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
8 Mutation Based on Directional Distributions . . . . . . . . . . . . . . . . . . 135
8.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
8.2 Directional Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.2.1 Von Mises–Fisher Distributions . . . . . . . . . . . . . . . . . . . . 139
8.3 Simulation Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
8.3.1 Conducting the Experiment . . . . . . . . . . . . . . . . . . . . . . . 146
8.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
8.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
Appendix A: Saddle Crossing Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Appendix B: Benchmark Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Abbreviations

DSM Discrete spectral measure

EP Evolutionary programming
EPN;a EP with mutation based on NSaSðrÞ distribution
EPI;a EP with mutation based on ISaSðrÞ distribution
EPS;a EP with mutation based on SaSUðrÞ distribution
ES Evolutionary strategy
ESSS Evolutionary search with soft selection
ESSSN;a ESSS with mutation based on NSaSðrÞ
ESSSI;a ESSS with mutation based on ISaSðrÞ
ESSSS;a ESSS with mutation based on SaSUðrÞ
ESSSa -DM ESSS with directional distribution Mðl; jÞ
ESTS Evolutionary search with tournament selection
ESTSN;a ESTS with mutation based on NSaSðrÞ
ESTSI;a ESTS with mutation based on ISaSðrÞ
ESTSS;a ESTS with mutation based on SaSUðrÞ
ESTSa -DM ESTS with directional distribution Mðl; jÞ
GA Genetic algorithm
GP Genetic programming
i.i.d. Independent and identical distribution
ðg þ kÞ ES ES with a succession of the best g individuals from joined base and
offspring populations
ðg þ kÞ ESN;a ðg þ kÞ ES with mutation based on NSaSðrÞ
ðg þ kÞ ESI;a ðg þ kÞ ES with mutation based on ISaSðrÞ
ðg þ kÞ ESS;a ðg þ kÞ ES with mutation based on SaSUðrÞ
ðg; kÞ ES ES with a succession of the best g individuals from k offspring
ðg; kÞ ESN;a ðg; kÞ ES with mutation based on NSaSðrÞ
ðg; kÞ ESI;a ðg; kÞ ES with mutation based on ISaSðrÞ
ðg; kÞ ESS;a ðg; kÞ ES with mutation based on SaSUðrÞ
SGA Simple GA

xi
Symbols

R Set of real numbers

G Genetic universe
D Phenotype universe
g Number of individuals in a population
PðtÞ Population in the t-th iteration
Pg ðGÞ Set of all possible populations, i.e., multisets of g elements
from G
atk k-th individual from the population PðtÞ
UðÞ Fitness function
A; B; . . .; X; Y; Z Random variables
A¼B
d Random variables A and B are of the same distribution
Xi:k i-th random variable of the ordered statistics of k variables
a Stability index (tail index)
b Skewness parameter
r Scale parameter
l Localization parameter
uðÞ Characteristic function
pðÞ Probability density function
EðÞ Expectation value
VarðÞ Variance
Uða; bÞ Continuous uniform distribution on (a, b)
Nðl; r2 Þ Normal (Gaussian) distribution
Cðl; rÞ Cauchy distribution
Levyðl; rÞ Lévy distribution
Sa ðr; b; lÞ a-stable distribution
SaSðrÞ Symmetric a-stable distribution
Cs Spectral measure
Nðl; CÞ Multidimensional elliptic Gaussian distribution
NSaSðrÞ Multidimensional non-isotropic symmetric a-stable distribution
ISaSðrÞ Multidimensional isotropic a-stable distribution

xiii
xiv Symbols

SaSUðrÞ Multidimensional isotropic distribution with an a-stable

generator
F General search space
MFðl; jÞ Von Mises–Fisher distribution
Mðl; jÞ Directional distribution
Chapter 1
Introduction

Practically any engineering activity, be it design, construction, modeling, control,

etc., sooner or later leads to the necessity of solving a set of optimization problems. In
practice, these problems usually appear to be multi-modal, sometimes multi-criteria
or non-stationary (changing during the searching process). Therefore, standard opti-
mization methods applied to solve them is inefficient. These techniques are usually
based on the so-called hard selection—new base points for further searching space
exploration are selected from the best points previously obtained. Using such proce-
dures, the solution sequence is usually trapped near to the first found local extremum,
without any possibility to localize others. So, the possibility of finding a global opti-
mum is strongly limited in this case. Methods, known from the literature, which try to
overcome this limitation can be divided into two classes: enumerative and stochastic
ones.
The first group is mainly dedicated to discrete and finite spaces, and these methods
usually consist in the examination of all solutions in order to designate the best one.
These approaches are, of course, of low efficacy, especially in the case of high
dimensional search spaces, where they cannot be realized in reasonable time, since a
full review is impossible to be performed in reasonable time here. Thus, methods of
heuristic search can be used (Schalkoff 1990). In the specific case, their effectiveness
strongly depends on the chosen heuristic strategy of domain searching.
Stochastic optimization techniques are usually applied to multi-modal and multi-
dimensional problems, for which the derivative is difficult or impossible to com-
pute. The advantage of these algorithms is their simple computer implementation.
However, they are time-consuming. In most of them, we can distinguish two phases:
global search and local search. The former determines starting points for local search.
Usually, these are randomly chosen from the domain. The goal of local search
is to find the local extremum, which is the attractor for the given starting point.
Among stochastic methods one can list several classes, e.g., pure random search
(Monte-Carlo methods), multiple random starts, clustering methods, random direc-
tion methods, or search concentration methods (Birge and Louveaux 1997; Zieliński
and Neumann 1983). The main disadvantage of stochastic methods lies in their
© Springer Nature Switzerland AG 2019 1
A. Obuchowicz, Stable Mutations for Evolutionary Algorithms,
Studies in Computational Intelligence 797,
https://doi.org/10.1007/978-3-030-01548-0_1
2 1 Introduction

chaotic manner, which does not take into consideration information contained in
previously evaluated points.
Particularly interesting stochastic methods of global extremum searching are
meta-heuristic ones (Trojanowski 2008), like simulated annealing, evolutionary algo-
rithms, particle swarm optimization and other algorithms based on the swarm idea,
etc. Special attention in this book is paid to evolutionary algorithms; however, the
presented results can be applied to other global optimization techniques in Rn . The
evolution is the natural way of development. Species acquire their properties and abil-
ities by natural selection, seemingly a blind process, which allows mainly well-fitted
individuals to survive and procreate. This mechanism permits transferring profitable
features to next generations, thus we have some kind of ‘intelligent’ selection. But
nature does not restrict itself to selecting only the best individuals in the population.
Weakly fitted individuals have a chance to introduce their descendant to the next gen-
eration, too. The descendants are often gifted with attributes unknown in the current
population, which can be useful in the future. Therefore, it is luring to introduce into
optimization techniques the soft selection rule instead of the hard one, i.e., there is
a possibility of choosing worse points as the base ones for further search. It appears
that soft selection accelerates the probability of escaping from a local optimum trap.
Soft selection is one of the basic properties of evolutionary algorithms, which,
thanks to this selection, are an effective computational intelligence technique ded-
icated to global optimization problems. The existing rich bibliography (Angeline
and Kinnear 1996; Arabas 2001; Bäck 1995; Bäck et al. 1997; Davis 1987; Fogel
1998; Dasgupta and Michalewicz 1997; Galar 1990; Goldberg 1989; Holland 1992;
Michalewicz 1996; Mitchel 1996; Schwefel 1995) confirms the thesis that a searching
process based on natural evolution is a very robust and effective global optimization
algorithm thanks to its adaptation abilities.
Most applications of evolutionary algorithms which use floating point represen-
tation of population individuals employ Gaussian mutation as a mutation operator
(Bäck and Schwefel 1993; Fogel et al. 1966; Fogel 1995; Galar 1985; Michalewicz
1996; Rechenberg 1965). A new individual x is obtained by adding a normally dis-
tributed random value to each entry of a selected parent y:

xi = yi + N (0, σi ), i = 1, . . . , n. (1.1)

The choice is usually justified by the central limit theorem. Mutations in nature
are caused by a variety of physical and chemical influences that are not identifiable
or measurable. These influences are considered to be independent and identically
distributed (i.i.d.). The general central limit theorem states that the only non-trivial
limit distribution of the random variable which is a normed sum of the series of i.i.d.
random factors is some α-stable distribution (called also the Lévy-stable distribution
or the heavy-tail distribution) (Fang et al. 1990; Nolan 2007; Samorodnitsky and
Taqqu 1994). If the Lindeberg condition is obeyed, i.e., the first two absolute moments
are finite, then the α-stable distribution reduces to the Gaussian one.
The main reason why α-stable distributions are not popular in the research com-
munity is the lack, in the general case, of the analytical form of the density function.
1 Introduction 3

Only three distributions from the α-stable family possess the analytical form of the
density function: the Gaussian, Cauchy and Lévy distributions. Fortunately, there
are algorithmic formulas for simulation of random α-stable variables (Nolan 2007)
as well as numerical algorithms for probability density functions, the cumulative
distribution and quantile determination (Nolan et al. 2001).
The Cauchy distribution was the first one, apart from Gaussian, to be applied to the
mutation operator in evolutionary algorithms (Bäck et al. 1997; Kappler 1996; Obu-
chowicz 2003a, b; Rudolph 1997; Yao and Liu 1996, 1997, 1999). However, there
is an unequivocal definition of the one-dimensional Cauchy distribution and two of
its multidimensional versions: isotropic and non-isotropic. The latter is constituted
by a random vector composed of independent one-dimensional Cauchy variables
(Fang et al. 1990; Obuchowicz 2003a). Thus, we can define non-isotropic Cauchy
mutation by the substitution in (1.1) of the random variable the Gaussian distribution
with the random variable of the Cauchy one. The course of the probability density
function of the one-dimensional Cauchy distribution is similar to the Gaussian one,
but the convergence of the Cauchy curve to the argument axis with the increasing
of the absolute value of the argument to the infinity is much slower than in the
case of the Gaussian one (α-stable distributions for low values of α are also called
heavy-tail distributions). Thus, the probability of macro-mutations (mutations which
locate the descendant far away from the basic point) significantly increases. Rudolph
(1997) presented a theoretical analysis of the local convergence of simple evolution-
ary strategies (1 + 1)ES and (1 + λ)ES with Gaussian, isotropic and non-isotropic
Cauchy mutations. He showed that the local convergence of evolutionary strategies
with Gaussian and isotropic Cauchy mutations is very similar. Non-isotropic Cauchy
mutation is characterized by significantly lower convergence to the local extremum.
Yao and Liu (1996, 1997, 1999) show that evolutionary programming and evolu-
tionary strategy algorithms with Cauchy mutation are very effective in comparison to
those with standard Gaussian mutation in the case of a wide class of 30-dimensional
global optimization tasks. This solution, however, has been obtained for benchmark
objective functions of special form. Most of them (e.g., Griewank, Rastringin or
Ackley functions—see Appendix B) possess local extrema localized in the nodes of
some hyper-cubic net whose axes are parallel to the axis of the reference frame. These
directions are significantly preferred by the multidimensional Cauchy distribution.
The influence of reference frame selection on the effectiveness of evolutionary algo-
rithms with Cauchy and Gaussian mutations in finding global optimization as well
as the algorithms’ ability of saddle crossing have been considered by Obuchowicz
(2003a, b).
The suggestion that the application of the α-stable distribution instead of the
Gaussian or Cauchy ones to evolutionary algorithms can be very attractive was first
proposed by Gutowski (2001). However, he considered only some properties of
α-stable distributions without any application to whatever evolutionary algorithm.
Such application has been simultaneously and independently presented and analyzed
by two research groups (Lee and Yao 2004; Obuchowicz and Pre˛tki 2004a). Lee
and Yao (2004) apply the evolutionary programming algorithm with non-isotropic
α-stable mutations to solving a set of 14 global optimization problems. Obuchowicz
4 1 Introduction

and Pre˛tki (2004a) study the influence of two little-known features—the so-called
symmetry effect and dead-surrounding effect—of non-isotropic α-stable mutation on
exploration and exploitation abilities of evolutionary algorithms. Hansen et al. (2006)
notice some limitations of heavy-tail distributions by analyzing certain properties of
isotropic and non-isotropic Cauchy distributions. They experimentally show that α-
stable mutations are only effective in the optimization of the multi-modal function
in research spaces of relatively low dimensions.
The goal of this book is to present a set of theoretical and experimental results
which describe features of the wide family of α-stable distributions, various methods
of their application to the mutation operator of evolutionary algorithms based on
a real-number representation of the individuals, and, most of all, to equip these
algorithms with features that enrich their effectiveness in solving multi-modal, multi-
dimensional global optimization problems.
This book is divided into seven main chapters preceded by this introduction and
supplemented at the end by a summary.
Basic terms of evolutionary algorithms, as well as their general outline, are intro-
duced in Chap. 2. Next, classical versions of the best-known representative of evo-
lutionary algorithms are described. Additionally, an evolutionary search with soft
selection algorithm is presented. This simple evolutionary algorithm is one of the
main algorithms used in simulating experiments described in further chapters of this
monograph.
Definitions of the random stable variable, as well as the random stable vector, are
included in Chap. 3. Moreover, foundations of the generator of pseudo-random vari-
ables of the α-stable distribution, as well as selected features of α-stable distributions,
are presented. Special attention is focused on the features which can suggest that the
application of α-stable distributions to the mutation operator of an evolutionary algo-
rithm can increase the efficacy of the searching process. One of the most important
theorems describes conditions of the existence of the expectation value of the first
value of the ordered statistics of the α-stable random variables series (Obuchow-
icz and Pre˛tki 2005). The main conclusion of this fact is that mutation based on the
heavy-tail distribution can be more effective in the localization of the local extremum
if only the parental individual has a possibility to create enough descendants. Chapter
3 contains also definitions of isotropic and non-isotropic random stable vectors, as
well as a description of the isotropic distribution with the α-stable generator. The
basic characteristics of the above-mentioned random vectors are discussed.
Non-isotropic stable mutations are the subject of Chap. 4. The analysis of the influ-
ence of the symmetry and dead surrounding effects on exploration and exploitation
abilities of evolutionary search is emphasized.
Application of isotropic stable mutations, described in Chap. 5, allows avoiding
problems with symmetry in a natural way. However, the dead surrounding effect still
influences the searching process. It is surprising that the range of the dead surrounding
decreases with the stability index α decreasing if isotropic stable mutation is used,
unlike in the non-isotropic case. Further sections of that chapter contain an analysis of
exploration and exploitation abilities of an evolutionary strategy with isotropic stable
mutation as well as the robustness of the evolutionary algorithm to the inaccuracy
1 Introduction 5

of selection of mutation control parameters. An attempt at scale parameter adaptation

strategy development for the chosen stability index is described in the last section of
the chapter.
Implementations of multidimensional stable distributions in global optimization
algorithms have so far been limited to two relatively simple techniques: the base
is mutated by adding to it a random vector composed of independent random sta-
ble variables or an isotropic random stable vector. This limitation does not exploit
many characteristics of α-stable distributions, which can turn out profitable during
the global optimum searching process. In order to achieve the possibility of model-
ing more complicated dependencies between decision variables, the application of
the discrete spectral measure to the generation of the wide class of random stable
vectors is proposed (Obuchowicz and Pre˛tki 2010). The main features of random sta-
ble vectors based on the discrete spectral measure are presented in Chap. 6. Using a
simple simulating experiment it is shown that, for a given objective function, includ-
ing dependencies between particular components of the random mutation vector
increases the effectiveness of the extremum searching process. An attempt at the
discrete spectral measure to the neighborhood of the current base point is shown at
the end of the chapter.
In order to avoid inconveniences resulting from the symmetry and dead surround-
ing effects, a new type of mutation operator is proposed in Chap. 7. A descendant is
obtained by a random vector with the α-stable generator added to the parent element.
This random vector can be statistically decomposed into a directional random vector
of a uniform distribution on the unit spherical surface and a α-stable random vari-
able which represents the distance between the parent element and the descendant
(Obuchowicz 2003b; Obuchowicz and Pre˛tki 2004b). The obtained multidimensional
distribution is isotropic, but, in the general case, it is not stable. The main advantage
of the mutation considered is the minimization of the dead surrounding range, which
is influenced only by the form of the objective function and is independent of the
chosen mutation distribution. The analysis of the effectiveness of evolutionary algo-
rithms with the mutation considered in local and global optimization tasks is given
in Chap. 7. Results of the analysis of these algorithms’ exploration and exploitation
abilities are also presented.
Undoubtedly, isotropic probabilistic models possess both advantages and disad-
vantages. A low number of parameters needed for their definition is a very impor-
tant advantage. Using them with stochastic optimization algorithms simplifies the
searching process. A potential heuristic adaptation strategy of parameters regarding
the current environment can be relatively simple. A low number of parameters is also
somewhat disadvantageous, which can have negative influence on optimization effi-
cacy. The isotropic probabilistic model allows taking into account the dependence
between an objective function and the Euclidian distance from a base point only.
If there exist stronger correlations between decision variables of the optimization
problem, then a stochastic searching process with mutation based on the isotropic
distribution can be extremely time consuming. In order to overcome these inconve-
niences, evolutionary algorithms with mutation based on the elliptic normal distribu-
tion have been proposed (Beyer and Schwefel 2002; Beyer and Arnold 2003; Hansen
6 1 Introduction

and Ostermeyer 2001). These algorithms are usually enriched with the mechanism
of mutation parameter adaptation. Then it is possible to fit the exploration model
to the currently explored optimization landscape. However, the mutation operator
based on the elliptical normal distribution allows modeling the relationship between
particular decision variables, but it possesses some important disadvantages. Both
directions (the one leading to the expected improvement and the opposite one) have
the same probability of selection. This fact leads to slow convergence of the searching
processes considered. Another serious problem, called the dimensionality curse, is
reflected in strong decreasing of the searching process with the increasing dimension-
ality of the searching space (Hansen et al. 2006; Pre˛tki and Obuchowicz 2006). The
choice of the spectral measure that prefers mutation in some improvement direction,
which can be identified using information contained in distributions of previously
verified populations in an evolutionary process, is an intuitive solution (Obuchow-
icz 2003b). Such directional distributions are proposed by Pre˛tki and Obuchowicz
(2006) and described in detail in Chap. 8.
Solutions set forth in this book for mutation operators based on α-stable distri-
butions have already been successfully applied. Isotropic mutation with the α-stable
generator, proposed in Chap. 7, has been successfully implemented in immune-based
algorithms (Trojanowski and Wierzchon 2009), multi-swarm quantum particles
(Trojanowski 2009) and differential evolution (Trojanowski et al. 2013) dedicated
for dynamic optimization problems. Isotropic stable mutation, described in Chap. 5,
has been applied the hierarchic genetic search system (HGS) (Obuchowicz 2015),
especially in the hierarchic evolutionary inverse solver (Obuchowicz and Smołka
2016).
I would like to express my sincere thanks to a number of people. First of all, I am
grateful to Professor Józef Korbicz for his continuous support and advice. I also wish
to thank Professor Marek Gutowski for the inspiration for this research area, as well
as Professors Jarosław Arabas, Roman Galar, Robert Schaefer and my son Adam for
their active interest in my research and many stimulating suggestions. A significant
part of results presented in this monograph have been obtained during several years
of cooperation with Doctor Przemysław Pre˛tki, who is a co-author of Chap. 5 of this
book and to whom I hereby express my special thanks and hope that the experience
gained during our cooperation will be useful in his professional career.
I also wish to express my deepest gratitude to my wife Beata for her everlasting
patience, understanding and support during my work on this book and throughout
all these years.

The research presented in this book has been partially funded by the National Science
Centre in Poland through the project no. 2015/17/B/ST7/03704.

Zielona Góra, May 2018 Andrzej Obuchowicz

References 7

References

Angeline, P., & Kinnear, K. E. (1996). Advances in genetic programming. Cambridge: MIT Press.
Arabas, J. (2001). Lectures on evolutionary algorithms. Warsaw (in Polish): WNT.
Bäck, T. (1995). Evolutionary algorithms in theory and practice. Oxford: Oxford University Press.
Bäck, T., & Schwefel, H.-P. (1993). An overview of evolutionary algorithms for parameter opti-
mization. Evolutionary Computation, 1(1), 1–23.
Bäck, T., Fogel, D. B., & Michalewicz, Z. (Eds.). (1997). Handbook of evolutionary computation.
New York: Institute of Physics Publishing and Oxford University Press.
Beyer, H. G., & Schwefel, H. P. (2002). Evolution strategies-a comprehensive introduction. Natural
Computing, 1(1), 3–52.
Beyer, H. G., & Arnold, D. V. (2003). Qualms regarding the optimality of cumulative path length
control in CSA/CMA-evolution strategies. Evolutionary Computation, 11(1), 19–28.
Birge, J., & Louveaux, F. (1997). Introduction to stochastic programming. New York: Springer.
Dasgupta, D., & Michalewicz, Z. (Eds.). (1997). Evolutionary algorithms for engineering applica-
tions. Berlin: Springer.
Davis, L. (Ed.). (1987). Genetic algorithms and simulated annealing. San Francisco: Morgan Kauf-
mann.
Fang, K.-T., Kotz, S., & Ng, K. W. (1990). Symmetric multivariate and related distributions. London:
Chapman and Hall.
Fogel, D. B. (1995). Evolutionary computation: Toward a new philosophy of machine intelligence.
New York: IEEE Press.
Fogel, D. B. (1998). Evolutionary computation: The fossil record. New York: IEEE Press.
Fogel, L. J., Owens, A. J., & Walsh, M. J. (1966). Artificial intelligence through simulated evolution.
New York: Wiley.
Galar, R. (1985). Handicapped individual in evolutionary processes. Biological Cybernetics, 51,
1–9.
Galar, R. (1990). Soft selection in random global adaptation in Rn . A biocybernetic model of
development. Wrocław (in Polish): Technical University of Wrocław Press.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Addison-
Wesley, Reading.
Gutowski, M. (2001). Lévy flights as an underlying mechanism for global optimization algorithms.
5th Conference on Evolutionary Algorithms and Global Optimization (pp. 79–86). Warsaw:
Warsaw University of Technology Press.
Hansen, N., & Ostermeyer, A. (2001). Completely derandomized self-adaptation in evolutionary
strategies. Evolutionary Computation, 9(2), 159–195.
Hansen, N., Gemperle, F., Auger, A., & Koumoutsakos, P. (2006). When do heavy-tail distributions
help? In T. Ph. Runarsson, H.-G Beyer, E. Burke, J. J. Merelo-Guervós, L.D. Whitley & X.
Yao (Eds.), Problem solving from nature (PPSN) IX (Vol. 4193, pp. 62–71). Lecture Notes in
Computer Science. Berlin: Springer.
Holland, J. H. (1992). Adaptation in natural and artificial systems. Cambridge: MIT Press.
Kappler, C. (1996). Are evolutionary algorithms improved by large mutation. In: H.-M. Voigt, W.
Ebeling, I. Rechenberg & H.-P. Schwefel (Eds.), Problem solving from nature (PPSN) IV (Vol.
1141, pp. 388–397). Lecture Notes in Computer Science. Berlin: Springer.
Lee, C. Y., & Yao, X. (2004). Evolutionary programming using mutation based on the Lévy prob-
ability distribution. IEEE Transactions on Evolutionary Computation, 8(1), 1–13.
Michalewicz, Z. (1996). Genetic algorithms + data structures = evolution programs. Heidelberg:
Springer.
Mitchel, M. (1996). An Introduction to genetic algorithms. Cambridge: MIT Press.
Nolan, J. P. (2007). Stable distributions-models for heavy tailed data. Boston: Birkhäuser.
Nolan, J. P., Panorska, A. K., & McCulloch, J. H. (2001). Estimation of stable spectral measures-
stable non-Gaussian models in finance and econometrics. Mathematical and Computer Modelling,
34(9), 1113–1122.
8 1 Introduction

Obuchowicz, A. (2003a). Multidimensional mutations in evolutionary algorithms based on real-

valued representation. International Journal of System Science, 34(7), 469–483.
Obuchowicz, A. (2003b). Evolutionary algorithms in global optimization and dynamic system
diagnosis. Zielona Góra: Lubuskie Scientific Society.
Obuchowicz, A., & Pre˛tki, P. (2004a). Evolutionary algorithms with α-stable mutations. In IEEE
4th International Conference on Intelligent Systems Design and Application, Budapest, Hungary,
CD-ROM.
Obuchowicz, A., & Pre˛tki, P. (2004b). Phenotypic evolution with mutation based on symmetric
α-stable distributions. International Journal on Applied Mathematics and Computer Science,
14(3), 289–316.
Obuchowicz, A., & Pre˛tki, P. (2005). Isotropic symmetric α-stable mutations for evolutionary
algorithms. In IEEE congress on evolutionary computation (pp. 404–410). Edinburgh, UK.
Obuchowicz, A., & Pre˛tki, P. (2010). Evolutionary algorithms with stable mutational based on
a discrete spectral measure. In L. Rutkowski, R. Scherer, R. Tadeusiewicz, L. A. Zadeh, & J.
M. Zurada (Eds.), Artificial intelligence and soft computing: Part II (Vol. 6114, pp. 181–188).
Lecture Notes on Artificial Intelligence. Berlin: Springer.
Obuchowicz, A. K., & Smołka, M. (2016). Application of α-stable mutation in hierarchic evolu-
tionary inverse solver. Journal on Computer Science, 17, 261–269.
Obuchowicz, A. K., Smołka, M., & Schaefer, R. (2015). Hierarchic genetic search with α-stable
mutation. In A. I. Esparcia-Alcázar, & A. M. Mora, (Eds), Applications of evolutionary compu-
tation (Vol. 9028, pp. 143–154). Lecture Notes in Computer Science. Berlin: Springer.
Pre˛tki P., & Obuchowicz A. (2006). Directional distributions and their application to evolutionary
algorithms. In L. Rutkowski, R. Scherer, R. Tadeusiewicz, L. A. Zadeh, & J. M. Zurada (Eds.),
Artificial intelligence and soft computing (Vol. 4029, pp. 440–449). Lecture Notes on Artificial
Intelligence. Berlin: Springer.
Rechenberg, I. (1965). Cybernetic solution path of an experimental problem. In Royal aircraft
establishment, Library Translation, 1122, Hants: Farnborough.
Rudolph, G. (1997). Local convergence rates of simple evolutionary algorithms with Cauchy muta-
tions. IEEE Transactions on Evolutionary Computation, 1(4), 249–258.
Samorodnitsky, G., & Taqqu, M. S. (1994). Stable non-Gaussian random processes. New York:
Chapman and Hall.
Schalkoff, R. J. (1990). Artificial intelligence: An engineering approach. New York: McGraw-Hill.
Schwefel, H.-P. (1995). Evolution and optimum seeking. New York: Wiley.
Trojanowski, K. (2008). Practical metaheuristics. Warsaw (in Polish): WIT Press.
Trojanowski, K. (2009). Properties of quantum particles in multi-swarm for dynamic optimization.
Fundamenta Informaticae, 95(2–3), 349–380.
Trojanowski, K., & Wierzchon, S. (2009). Immune-based algorithms for dynamic optimization.
Information Sciences, 179, 1495–1515.
Trojanowski K., Raciborski, M., & Kaczynski, P. (2013). Adaptive differential evolution with hybrid
rules of perturbation for dynamic optimization. In K. Madani, A. Dourado, A. Rosa, & J. Filipe
(Eds.), Computational intelligence (Vol. 465, pp. 69–83). Studies in Computational Intelligence.
Berlin: Springer.
Yao, X., & Liu, Y. (1996). Fast evolutionary programming. 5th Annual Conference on Evolutionary
Programming (pp. 419–429). Cambridge: MIT Press.
Yao, X., & Liu, Y. (1997). Fast evolutionary strategies. Control. Cybernetics, 26(3), 467–496.
Yao, X., & Liu, Y. (1999). Evolutionary programming made faster. IEEE Transactions on Evolu-
tionary Computation, 3(2), 82–102.
Zieliński, R. A., & Neumann, P. (1983). Stochastic methods of the function minimum searching.
Berlin: Springer.
Chapter 2
Foundations of Evolutionary Algorithms

Evolutionary algorithms are a broad class of stochastic adaptation algorithms inspired

by biological evolution—the process that allows populations of organisms to adapt
to their surrounding environment. The concept of evolution was introduced in the
19th century by Charles Darwin and Johann Gregor Mendel and, complemented with
further details, is still widely acknowledged as valid.
In 1859, Darwin published his theory of natural selection, or survival of the fittest.
The idea behand it is as follows: not every organism can be kept, only those which
can adapt and win the competition for food and shelter are able to survive. Almost at
the same time (1865) Mendel published a short monograph about experiments with
plant hybridization. He observed how traits of different parents are combined in a
descendant by sexual reproduction. Darwinian evolutionary theory and Mendel’s
investigations of heredity in plants became foundations of evolutionary search
methods.
The structure and properties of evolutionary algorithms are discussed in several
books (Angeline and Kinnear 1996; Arabas 2001; Bäck 1995; Bäck et al. 1997; Das-
gupta and Michalewicz 1997; Davis 1987; Fogel 1998; Galar 1990; Goldberg 1989;
Holland 1992; Michalewicz 1996; Mitchel 1996; Schaefer 2007; Schwefel 1995).
Papers concerned with evolutionary computation are published in many scientific
journals. There are at least 20 international conferences closely connected with evo-
lutionary methods. Due to the large number of available publications, it is impossible
to present all of the different evolutionary algorithms and their components, whose
authors tried to improve algorithm efficiency in the case of a given problem to be
solved. A relatively wide glance at population methods applied to the global opti-
mization problem can be found in the monographs by Arabas (2001), Bäck et al.
(1997) and Schaefer (2007). In this chapter, the main components of evolutionary
algorithms are recalled and their various basic forms are briefly discussed.

© Springer Nature Switzerland AG 2019 9

A. Obuchowicz, Stable Mutations for Evolutionary Algorithms,
Studies in Computational Intelligence 797,
https://doi.org/10.1007/978-3-030-01548-0_2
10 2 Foundations of Evolutionary Algorithms

2.1 Basic Concepts

In nature, individuals in a population compete with one another for resources such as
food, water, and shelter. Also, members of the same species often compete to attract a
mate. The individuals which are most successful in surviving and attracting mates will
have relatively larger numbers of descendants. Poorly performing individuals will
produce few or even no descendant at all. This means that the information (genes),
slightly mutated, from the highly adapted individuals will spread to an increasing
number of individuals in each successive generation. In this way, species evolve to
become more and more well-suited to their environment.
In order to describe a general outline of the evolutionary algorithm, let us introduce
some useful concepts and notations (Atmar 1992; Fogel 1999; Schaefer 2007). An
individual (a sample solution for some optimization or adaptation task) is represented
by the genotype (an underlying genetic coding) a ∈ G , which is an element of some
genetic universum G . A population P ∈ Pη (G ) is some multiset of η elements from
G . Pη (G ) represent a set of all possible such multisets. The environment delivers
quality information (fitness value) of the individual dependent on its phenotype (the
manner of response contained in the behavior, physiology and morphology of the
organism). The fitness function
Φ:D →R (2.1)

is defined on a phenotype space D. So, each individual can be viewed as a duality

of its genotype and phenotype, and so some decoding function, epigenesis,

ξ : G → D  ⊂ D, (2.2)

is needed.
At the beginning, an initial population P(0) is generated (in a random or determin-
istic manner) and evaluated, i.e., the fitness function is calculated for each element of
P(0) (Table 2.1). After initiation, the randomized processes of reproduction, recom-
bination, mutation and succession are iteratively repeated until a given termination
criterion,
ι : Pη (G ) → {true, false}, (2.3)

is satisfied.
Reproduction,
p
sθp : Pη (G ) → Pη (G ), (2.4)

called also preselection, is a randomized process (deterministic in some algorithms)

of selection of η parents from η individuals of the current population. This process
is controlled by a set θp of parameters.
2.1 Basic Concepts 11

Table 2.1 General outline of an evolutionary algorithm

I. Initiation
A. Random generation of the first population
 
P(0) = a0k | k = 1, 2, . . . , η
B. Evaluation
      
P(0) → Φ P(0) = φk0 = Φ ξ a0k | k = 1, 2, . . . , η
C. t = 0
II. Repeat
A. Reproduction
p    
P  (t) = sθp P(t) = a tk | k = 1, 2, . . . , η
B. Recombination
   
P  (t) = rθr P  (t) = a tk | k = 1, 2, . . . , η
C. Mutation
   
P  (t) = mθm P  (t) = a tk | k = 1, 2, . . . , η
D. Evaluation
      
P  (t) → Φ P  (t) = φkt = Φ ξ a tk | k = 1, 2, . . . , η
E. Succession
   
P(t + 1) = sθnn P(t) ∪ P  (t) = at+1 k | k = 1, 2, . . . , η
F. t = t + 1
  
Until ι P(t) = true

Recombination,
rθr : Pη (G ) → Pη (G ), (2.5)

is a mechanism (omitted in some realizations), controlled by additional parameters θr ,

which allows the mixing of parental information while passing it to their descendants.

Mutation
mθm : Pη (G ) → Pη (G ), (2.6)

introduces innovation into current descendants, θm is again a set of control parameters.

Succession,
sθnn : Pη (G ) × Pη (G ) → Pη (G ), (2.7)

also called postselection, is applied to choose a new generation of individuals from

parents and descendants.
p
Considering reproduction sθp , recombination rθr , mutation mθm and succession sθnn
operators, it is worth noting that they are random operators in the general case. For the
purpose of analysis of these operators and their influence on an evolutionary process,
target sets should be defined as some probabilistic space. But, if they are treated as
elements of a particular algorithm, in which given realizations of the random variables
12 2 Foundations of Evolutionary Algorithms

are obtained as their result, we assume that the target sets are the Cartesian product
of some number of genotype universes G .

2.2 Standard Evolutionary Algorithms

Apart from similarities among various evolutionary computation techniques, there

are also many differences. It is generally accepted that any evolutionary algorithm to
solve a problem must have five basic components (Davis 1987; Michalewicz 1999):
• a representation of solutions to the problem,
• a way to create an initial population of solutions,
• an evaluation function, rating solutions in terms of their fitness,
• selection and variation operators that alter the composition of children during
reproduction and mutation,
• values for the parameters (population size, probabilities of applying variation oper-
ators, etc.).
The variety of possible realizations of the above-mentioned components produces
very large variants of evolutionary algorithms, which are usually specialized and
dedicated to a particular optimization or adaptation task.
The duality of the genotype and the phenotype suggests two main approaches to
simulated evolution dedicated to global optimization problems in Rn : genotypic and
phenotypic simulations (Fogel 1999). The former are focused on genetic structures.
The candidate solutions are described as being analogous to chromosomes. The entire
searching process is provided in the genotype space G . However, in order to calculate
an individual’s fitness, its chromosome must be decoded to its phenotype. Two main
streams of instances of such evolutionary algorithms can nowadays be identified:
• genetic algorithms (GAs) (De Jong 1975; Goldberg 1989; Grefenstette 1986; Hol-
land 1975; Michalewicz 1996),
• genetic programming (GP) (Kinnear 1994; Koza 1992).
In phenotypic simulations, attention is focused on the behaviors of the candidate
solutions in a population. All searching operations, i.e., selection, reproduction and
mutation, are constructed in the phenotype space D. This type of simulation char-
acterizes a strong behavioral link between a parent and its descendant. Nowadays,
there are two main streams of instances of ‘phenotypic’ evolutionary algorithms:
• evolutionary programming (EP) (Fogel et al. 1966, 1991; Fogel 1999; Yao and
Liu 1999),
• evolutionary strategies (ESs) (Rechenberg 1965; Schwefel 1981).
Besides the above-mentioned algorithms, one more phenotypic model is of crucial
importance in the research described in this book. This is the evolutionary search
with soft selection (ESSS) algorithm (Galar 1985), which is a simplified version of
the ES. The basic ideas behind GA, GP, EP and ES algorithms are presented below.
The ESSS algorithm is the subject of the next section.
2.2 Standard Evolutionary Algorithms 13

2.2.1 Genetic Algorithms

GAs are probably the best known evolutionary algorithms, receiving remarkable
attention all over the world. The basic principles of GAs were first laid down rigor-
ously by Holland (1975), and are well described in many texts (e.g. Bäck and Schwe-
fel 1993; Beasley et al. 1993a, b Dasgupta and Michalewicz 1997; Davis 1987, 1991;
Goldberg 1989; Grefenstette 1986, 1990, 1990; Michalewicz 1996).
The previously proposed forms of GAs (De Jong 1975; Holland 1975) operate
on binary strings of fixed length l, i.e., the genotype space G is an l-dimensional
Hamming cube G = {0, 1}l . A GA with the binary representation of the individual
is a natural technique of solving discrete problems, especially in the case of finite
cardinality of possible solutions. Such a problem can be transformed to a pseudo-
Boolean fitness function, where the GA can be used directly. In the case of continuous
domains of optimization problems, the function ζ : D → G that encodes the vari-
ables of the given problem into a bit string (the so-called chromosome) is needed.
The encoding function ζ is non-invertible and there does not exist the inverse func-
tion ζ −1 . A decoding function ξ : G → Dl ⊂ D generates only 2l representatives of
solutions. This is a strong limitation of GAs. Moreover, it is worth noting that, taking
into account different topologies of genotype and phenotype spaces, the transforma-
tion ξ : D → G should fulfil additional, very difficult to obtain, conditions, which
prevent occurence of a new local optimum in G in comparison to D (Arabas 2001).
There are many known GA versions in the literature. In this chapter the version
described by Arabas (2001) is presented. The parent selection sp is carried out by
the so-called proportional method (roulette method):
 h 
    l=1 φl
t
s P(t) = ah1 , ah2 , . . . , ahη :
p
hk = min h : η > χk , (2.8)
l=1 φl
t

where {χk = U (0, 1) | k = 1, 2, . . . , η} are uniformly distributed, independent ran-

dom numbers from the interval [0, 1). In this type of selection, the probability that a
given chromosome will be chosen as a parent is proportional to its fitness. Because
sampling is carried out with returns, it can be expected that well-fitted individuals
insert a few of their copies in the temporary population P  (t).
After the reproduction, chromosomes from P  (t) are recombined using the so-
called one-point crossover operation. Chromosomes from P  (t) are joined into pairs.
The decision that a given pair will be recombined is made with the given probability
θr . If the decision is positive, an ith position in the chromosome is randomly chosen
and the information from the position (i + 1) to the end of chromosomes is exchanged
in the pair:    
(a1 , a2 , . . . , al ) (a1 , . . . , ai , bi+1 , . . . , bl )
→ .
(b1 , b2 , . . . , bl ) (b1 , . . . , bi , ai+1 , . . . , al )

The newly obtained temporary population P  (t) is mutated. The individuals’

mutation mθm is performed separately for each bit in a chromosome. The bit value is
14 2 Foundations of Evolutionary Algorithms

changed to the opposite one with the given probability θm . The obtained population
is the population of a new generation.
Historically, the first attempt at a formal description of the asymptotic charac-
teristics of the GA was made by Holland (1975). The combined effect of selection,
crossover and mutation gives the so-called reproductive schema growth equation
(Schaefer 2007):

Φ(S, t) δ(S)  o(S)

E[η(S, t + 1)] ≥ η(S, t) 1 − θr 1 − θm , (2.9)
Φ̄(t) l − 1

where S is a schema defined as a string of l symbols from an alphabet Σ = {0, 1, },

each schema represents all strings which match it on all positions other than ‘ ’,
η(S, t) denotes the number of strings in a population at the time t matched by schema
S, E[·] is a symbol of an expectation value, δ(S) is the defining length of the schema
S (the distance between the first and the last fixed string position), o(S) denotes the
order of the schema S (the number of 0 and 1 positions presented in the schema),
Φ(S, t) is defined as the average fitness of the all strings in the population at the time
t matched by the schema S, and Φ̄(t) is the average fitness taken over all individuals
in the population at the time t.
Equation (2.9) tells us about the expected number of strings matching a schema
S in the next generation as a function of the current number of strings matching the
schema, the relative fitness of the schema, as well as its defining length and order.
It is clear that above-average schemata with a short defining length and a low order
would still be sampled at exponentially increased rates.
The above approach, often criticized (see Schaefer 2007; Grefenstette 1993), can
be treated as an attempt at the evaluation of numerical improvement of population
quality (Whitley 1994). Vose (1999) proves, under some additional conditions, that
a Markov process, which models genetic algorithm processing, is ergodic. This fact
implies asymptotic correctness in the probabilistic sense and the asymptotic guaran-
tee of success (Schaefer 2007).

2.2.2 Genetic Programming

Many trends in SGA development are connected with the change of an individual
representation. One of them deserves particular attention: each individual is a tree
(Koza 1992). This small change in the GA gives evolutionary techniques the possi-
bility of solving problems which have not been tackled yet. This type of GA is called
genetic programming (GP).
Two sets are needed to be defined before GP starts: the set of terms T and the set
of operators F . In the initiation step, the population of trees is randomly chosen. For
each tree, leaves are chosen from the set T and other nodes are chosen from the set
F . Depending on definitions T and F , a tree can represent a polyadic function, a
2.2 Standard Evolutionary Algorithms 15

Fig. 2.1 Sample of a tree which represents the function f (x, y, z) = yz + sin(2π x)

crossover

mutation

random subtree

Fig. 2.2 Genetic operators for GP

logical sentence or a part of a programming code in a given programming language.

Figure 2.1 presents a sample tree for T = {x, y, z, 2, π } and F = {∗, +, −, sin}.
The new type of individual representation needs new definitions of crossover and
mutation operators, both explained in Fig. 2.2.
16 2 Foundations of Evolutionary Algorithms

2.2.3 Evolutionary Programming

Evolutionary programming resides in the ‘phenotypic’ category of simulations. It

was devised by Fogel et al. (1966) in the mid-1960s for the evolution of finite state
machines in order to solve prediction tasks. The environment was described as a
sequence of symbols (from a finite alphabet), and a finite state machine had to create
a new symbol. The output symbol had to maximize some profit function, which
was a measure of prediction accuracy. There are no reproduction or recombination
operators. Each machine of the current population generates a descendant by random
mutation. There are five possible modes of random mutation that naturally result from
the description of the finite state machine: changing an output symbol, changing a
state transition, adding a state, deleting a state, or changing the initial state. Mutation
is chosen with respect to a uniform distribution. The best half number of parents and
descendant are selected to survive.
EP was extended by Fogel (1991, 1992) to work on real-valued object variables
based on normally distributed mutations. This algorithm was called meta-EP (Fogel
et al. 1991) or classical EP (CEP) (Yao and Liu 1999). The description shown in
Table 2.2 is based on the work of Bäck and Schwefel (1993), as well as Yao and Liu
(1999).
In meta-EP, an individual is represented by a pair a = (x, σ ), where x ∈ Rn is
a real-valued phenotype while σ ∈ Rn+ is a self-adapted standard deviation vec-
tor for Gaussian mutation. For initialization, EP assumes bounded initial domains
Ωx = ni=1 [ui , vi ] ⊂ Rn and Ωσ = ni=1 [0, c] ⊂ Rn+ , with ui < vi and c > 0. How-
ever, the search domain is extended to Rn × Rn+ during algorithm processing. As a
mutation operator, Gaussian mutation with a standard deviation vector ascribed to
an individual is used. All elements in the current population are mutated. Individuals
from both parent and descendant populations participate in the process of selecting
a new generation. For each individual ak , q individuals are chosen at random from
P(t) ∪ P  (t) and compared to ak with respect to their fitness values. wk is the number
of q individuals worse than ak . η individuals having the highest score wk are selected
from 2η parents and descendants to form a new population P(t + 1).
The analysis of the classical EP algorithm (Fogel 1992) gives a proof of the global
convergence with probability one for the resulting algorithm, and the result is derived
from defining a Markov chain over the discrete state space that is obtained from a
reduction of the abstract search space Rn to the finite set of numbers representable
on a digital computer.

2.2.4 Evolutionary Strategies

The other well-known ‘phenotypical’ algorithms are evolutionary strategies, which

were introduced in the mid-1960s by Rechenberg (1965) and further developed by
Schwefel (1981). The description of the ES presented in this subsection is based on
the article by Bäck and Schwefel (1993). The general form of the ES relies on the
2.2 Standard Evolutionary Algorithms 17

Table 2.2 Outline of the EP algorithm

I. Initialize
A. Random generation
   
P(0) = a0k = x0k , σ k (0) | k = 1, 2, . . . , η
x0k = RANDOM (Ωx ), σ 0k = RANDOM (Ωσ ),
Ωx ⊂ Rn , Ωσ ⊂ Rn+
B. Evaluation
     
P(0) → Φ P(0) = φk0 = Φ x0k | k = 1, 2, . . . , η
C. t = 1
II. Repeat
A. Mutation
   
P  (t) = mτ,τ  P(t) = a tk | k = 1, 2, . . . , η
 
x tki = xki
t + σ t N (0, 1), σ  t = σ t exp τ  N (0, 1) + τ N (0, 1) ,
ki i ki ki i
i = 1, 2, . . . , n,
where N (0, 1) denotes a normally distributed one-dimensional random
number with mean zero and standard deviation one, Ni (0, 1) indicates
that the random number is generated anew for each component i
B. Evaluation
     
P  (t) → Φ P  (t) = φ  tk = Φ x tk | k = 1, 2, . . . , η
C. Succession
   
P(t + 1) = sθnn P(t) ∪ P  (t) = at+1 k | k = 1, 2, . . . , η
∀atk ∈ P(t) ∪ P  (t),
   
atk → atkl = RANDOM P(t) ∪ P  (t) | l = 1, 2, . . . , q ,
q   0 for α < 0
wkt = l=1 θ Φ(xtk ) − Φ(xtkl ) , θ(α) = ,
1 for α ≥ 0
P(t + 1) ← η individuals with the best scorewkt
D. t = t + 1
  
Until ι P(t) = true

individual representation in the form of a pair a = (x, C), where x ∈ Rn is a point

in a searching space, and the fitness value of the individual a is calculated directly
from the objective function: Φ(a) = f (x). C is the covariance matrix for the general
n-dimensional normal distribution N(0, C), having the probability density function

1 1
p(z) = exp − zT C−1 z , (2.10)
(2π )n det(C) 2

where z ∈ Rn . To assure positive-definiteness

 of C, it is described by two vectors:
the vector of standard deviations σ cii = σi2 and that of rotation angles α(cij =
 2 
1
2
σi − σj2 tan 2αij ). Thus, a = (x, σ , α) is used to denote a complete individual.
There is no separate operation of selection of parents in ESs; this selection is
strongly connected with the recombination mechanism. Different recombination
Another random document with
no related content on Scribd:
fact that she herself was the cause of their silence. Luke indeed did not notice
that Rachel was not talking, or that he himself had very little opportunity of
doing so, for he was naturally a silent man, having contracted the habit from
having so talkative a mother.

Happily after lunch, Mrs. Greville had to go to some parish engagement so

that Rachel and Luke had their chance of a talk; and finally Luke was called
off to see someone and Rachel had the Bishop to herself.

The talk did her good, specially as she made him laugh over the matter of the
two pies.

"Now that lunch is over," she said laughingly, "I am thankful that I changed
them. I believe it would have given my mother-in-law a terrible shock if she
had found out that I neither wanted nor liked her pie. And perhaps it would
have ended in an estrangement between me and Luke as he would have
probably heard of it, and I am quite sure he would never have understood.
And fancy! All because of a pie! How silly and small I am."

CHAPTER IX.
GWEN.

"I expect that Gwen has been in one of her naughty moods," said Rachel, as
she passed the letter she had received from her mother to Luke when they
were at breakfast. "I was the only one who could do anything with her."

"I see that she is coming instead of Sybil."

"Yes, and I am sure that is the reason. I shall love to have her."

"I think I shall be somewhat afraid of that young person," said Luke with a
laugh. "She is one of the independent kind I noticed at our wedding."

"She is a darling, and I know you will love her. But I own that at times she is
an enfant terrible, one never knows what she is going to say next. One thing,
however, we may be sure of, she is absolutely true, and says what she really
thinks. You must prepare for the worst," she added, laughing, "and you must
overlook her faults. I shall not forgive you if you don't love her. To me she is a
most fascinating little thing."

And Gwen arrived the next day. She was a girl of fifteen, tall and slim, not
exactly pretty; but there was a charm about her that could not be denied, and
Rachel, as she met her at the station, could not help hugging her. She was a
bit of home, fresh and sweet; and carried about with her the atmosphere of
golden cornfields and scented hedges. Rachel had not seen anything so fresh
and full of life since leaving home.

On the other hand Gwen had never seen anything like the darkness and dirt
of the town through which she was passing to Rachel's home. She grew silent
as they drove through the streets.

Rachel wondered what she was thinking of, and tried to distract her attention
by questions about her mother and sister; but only received short answers
and in an absent tone of voice.

At last they reached number 8 Wentworth Road.

"Is this it?" asked Gwen incredulously.

"Yes. It is not pretty, but I have tried to make it nice inside; and have quite got
to love it," answered Rachel. She was a little distressed at Gwen's tone of
voice.

When they had given directions to the cabman to leave the luggage a few
doors further up, Rachel took her sister over the house, and they finally settled
down by the drawing-room fire, as the evenings were beginning to get chilly;
though they had not begun fires, Rachel was bent on having one on the day of
Gwen's arrival.

Gwen drew her chair up almost into the fender, and then clasping her hands
behind her head said, "Now I will answer your questions properly about home.
I really couldn't do so in that awful cab and passing through the town. What a
place it is!"

"I suppose it strikes you as very uninviting, but I have got so used to it that I
hardly notice its deformities."

"Well it is time that someone should come and spy out the land," said Gwen. "I
am sure that Mother has no idea of your surroundings."

Rachel laughed.

"Well don't you go and make the worst of them to her," she said. "I have
purposely not enlarged on the subject, as I did not wish to worry her. Besides,
she would imagine that I was not happy, which would be very far from the
truth. I would far rather live in an ugly dirty town with Luke than in the most
beautiful country in the world without him. When you are a little older, Gwen,
you will understand that."

"No I shan't. No man in the world would make up to me for the country. I
should simply die if I had to live here," she added, looking round the tiny room.
"In fact I can't imagine a really unselfish man asking such a sacrifice from the
girl he loves best in all the world."

Rachel laughed merrily. Gwen had got on to her favourite theme, the
selfishness of men. She was always harping on that subject, Rachel
remembered, at Heathland.

"Well, let us leave that and tell me of home," she said, as she was hungering
for news. Then she suddenly drew Gwen's chair closer to her.

"You dear little thing," she said, smoothing her hair tenderly. "How glad I am to
have you. I'm afraid, however, that you have come because you have been
troublesome at home. Is that so?"

"I've come to spy out the land," answered Gwen with a mischievous smile;
"and it's high time."

"Don't be silly, tell me about Mother and Sybil."

"Mother is a dear and lovely as ever. I wish I had not made her cry last week. I
own I was horrid."

"Oh Gwen! You don't know what it is to be without Mother."

"I'm thankful I don't," said Gwen energetically. "If Mother had seen this place
before you married Luke she would never have let you come. By-the-bye, I
suppose that funny little creature that opened the door for us is not Polly who
you write about?"

"Yes, she is Polly. We are great friends."

"But she is not the only one?"

"Of course she is. Why you don't suppose this tiny house requires more than
one servant do you?"

"But that minute specimen cannot do all that is needed by herself."

"Of course not. I help her. Now don't be stupid Gwen; tell me some more
about home."

Gwen shut her mouth indicative of intense disapproval for a moment; then she
began to talk of Heathland; and Rachel listening, could almost feel the wind
blowing over the moors, and see the hedges just touched with hoarfrost in the
morning. She pictured her Mother walking about the garden with her pretty
soft shady hat which they all thought suited her so well, or lying on the cane
sofa in the verandah speaking to the old gardener in her low musical voice.
The vision of her was so vivid that the tears rushed into Rachel's eyes, and
would have fallen had it not been for Gwen's presence. She was determined
that the tiresome child should not have any excuse for supposing she was not
as happy as a queen.

It was at supper that Luke met the 'young person' as he called her, and had to
confess to himself that he was more alarmed at her than she was of him.

Gwen was afraid of no-one, specially of a mere man, as she had made up her
mind that they were a set of selfish human beings who needed to be taught
what was really required of them, and that one woman was worth ten men;
specially such a woman as Rachel whom she loved devotedly.

In fact the selfishness of Luke had chiefly consisted in her mind in taking her
favourite sister away from her. She knew little but that about him, and though
she had been sent away from home in order that a change might help her to
get rid of her very tiresome mood, she preferred looking upon her visit to
Rachel in the light of a spy.
Was Luke worthy of her? Had he made her comfortable? Did he look well after
her? These were the questions that she intended answering during her visit,
and taking the answer back to her mother and sister.

But she soon found that it would not do to make the object of her visit too plain
to Rachel, as the latter showed signs of being vexed; and she might defeat
her own plan. So when Luke came in to supper she was on her best
behaviour, though at times she could not prevent her lips curling at one or two
of his remarks. It seemed to her that he was wrapped up in his own interests
and noticed nothing else. She did not realise the immense importance of his
interests which were centred in his work.

"We must try and give Gwen a little amusement," said Rachel next morning
before her sister arrived for breakfast, "or we shan't keep her with us. Don't
you think we could take her to the wood this afternoon?"

He told her by all means to go to Deasely Woods, but that he had work which
could not be neglected.

To be in the woods again with Gwen satisfied a longing of Rachel's heart.

They left the dullness and dirt of Trowsby behind them, and wandered among
the trees, treading on the soft carpet of fallen leaves and inhaling the scent of
the damp earth.

"How delicious," Rachel exclaimed.

"Do you often come here?" asked Gwen. She knew what the answer would
be.

"No, Luke can't afford the time. You see the calls on his time are endless in
such a parish."

"Bother the parish!" said Gwen.

"No, no, you must not say that. I don't think you quite realise that a
clergyman's life is quite different to that of other people. You would not
approve of a doctor neglecting his patients for pleasure. Well a clergyman is a
physician of souls. And after all souls are more important than bodies."

"I don't know anything about souls," said Gwen.

"Of course you do, don't talk nonsense."

"No, I don't. I don't think I am sure that we have souls. But I am not peculiar in
this. The papers and books are full of doubts of all sorts."

"But my dear child, why do you read such books? We want to build ourselves
up in our most holy faith, and not to read all the views on the other side. How
do you see these books?"

"I find them in the library. Sometimes I wish I had not read them, but you know
I read everything I can get hold of."

Rachel made up her mind to ask Luke to have a talk with Gwen. She was very
distressed at what she told her.

"Luke says that we must not be surprised at all the doubts and strange
theories that are about just now, as he believes we are living in the last days
and must expect the devil to be extra busy. I am sure he is right."

"Don't let us talk of the last days," said Gwen, "but enjoy the country while we
have a chance. You must pine for it in that horrid place."

"I am too busy to think much about it," said Rachel, and she added, "when I
do I turn my thoughts to Luke, and feel how much I have to be thankful for in
having him."

Gwen laughed a little unbelievingly; and on returning to Trowsby, she felt she
could not endure more than a few days in it although her favourite sister lived
there. Of course she helped Rachel with the household work, and made fun of
it; but she hated it for all that, and could not understand how Rachel could
endure it after her life at home. She studied Luke attentively and critically;
nothing escaped her, and a day or two before she left, he heard a knock at his
study door and on opening it found Gwen facing him.

"I want to talk about something very important," she said.

Luke was in the midst of writing a paper to be read at a clerical meeting, and
was sorry to be interrupted; but he invited her in with a smile and drew up a
chair for her. She seated herself and then looked up at him gravely. He
wondered what was coming. Gwen's expression of face was severe.

"I suppose you know how unwell Rachel is," she began.

"Unwell?" said Luke startled.

"Yes, she is quite different to what she was at home. She has lost all her
spirits and looks. Do you mean to say you have not noticed?"

"No, certainly I have not," said Luke. "She is always very bright."

"That's just like a man," said Gwen scornfully. "They never notice when their
wives look ill. They are all alike. Rachel is working far too hard, it will wear her
out."

Luke rose greatly concerned and leant against the mantle piece looking down
at his severe young judge, anxiously.

"Are you sure?" he asked.

"Perfectly sure. She can't stand this life; having never been used to it. It is all
very well for girls who have been taught how to do things. Some of them quite
like it. But Rachel has never been taught and it is killing her, slowly."

Luke leant his head on his hand which rested on the mantle piece and fixed
sad eyes on the girl. He was too perplexed and worried to speak.

"Rachel positively slaves for you," continued Gwen unmercifully, "but you don't
see or notice. Why only the other evening she carried a heavy coal scuttle into
the dining-room and you were so deep in your paper that you never saw. You
don't see or know half that goes on. But all men are alike. Certainly from all I
see of married life I never intend to marry; if I do I am determined to be an old
man's darling rather than a young man's slave."

Worried as Luke was, he could not resist a smile, as the thought crossed his
mind that Gwen would probably never have the chance of being either. He
could not imagine any man falling in love with such an audacious young
person. His smile however quickly disappeared as Gwen said:

"I suppose you love her still?"

"Love her!" He grew white and his eyes flashed so fiercely that Gwen for a
moment quailed.

"Child, you don't know what you are talking about," he said, and stood looking
at her with amazement and anger.

"Well you don't seem to. At home when you were engaged I now and then
intercepted glances between you that almost reconciled me to losing my
favourite sister, as I was assured by them that she was all the world to you,
and that you would take care of her. But now you scarcely seem aware of her
presence, and she might be a piece of furniture for all the attention she gets. I
can't think how she can bear it."

Had Luke not remembered his calling, and had he not been accustomed to
keep himself in check, he would have shaken the girl who had constituted
herself as his judge. As it was he went towards his writing table and began
arranging his papers, saying:

"I am sorry I cannot spare you any more time. When you are a little older you
will understand more of the meaning of love," he added looking at her gravely,
"that a man and his wife are so one that it is perfectly unnecessary for them to
remind each other of their existence or of their love for one another. Happily
for me Rachel understands and absolutely trusts me."

Gwen rose.

"But that does not explain about the coal scuttle business," she said, "I do
hope Luke," she added, "that you will take care of her. She has given up
everything for you."

Luke held the door open for Gwen politely, and was silent. Then he locked it
after her and sitting down by his desk tried to write. But he found this was
impossible. He felt all on edge. How dared the child talk as she did; but when
his irritation had subsided the remembrance of her words fell like lead on his
heart. Was it a fact Rachel had lost her spirits and that Gwen saw a real
change in her since her marriage?

He began pacing up and down his study while a terrible anxiety weighed upon
his mind. Was she not happy? Did she regret the step she had taken? And the
fear that every now and then had attacked him as to the rightfulness of taking
her away from her happy home, gained ground.

He could laugh off the ridiculous fuss Gwen had made about the coal scuttle.
Of course he had been quite unaware of Rachel carrying the heavy weight
across the room. He was able so to concentrate his attention on what he was
reading that he seldom noticed what was passing round about him, unless he
was trying to solve some difficult problem, when every sound disturbed him.
But he was so used to reading while Rachel moved about the room that he
had noticed nothing till he remembered Gwen had called out to him, "Luke,
don't you see that Rachel is carrying the scuttle?" when he had risen at once,
but too late.
That it proved in the very least that he was not careful of her he would not
admit for a moment. Neither did he pay any heed to Gwen's ridiculous fancy
that because he was not always showing his devotion to his wife by his
glances, his affection had waned. These ideas did not trouble him; but the fact
that Gwen had noticed Rachel was looking and had lost her spirits was quite
another thing, and it worried him exceedingly.

Meanwhile Gwen had gone into the drawing-room where she found Rachel
writing home. She turned round at the sound of her entrance.

"What have you been talking to Luke about?" she said a little anxiously. "You
have been a long time in the study."

"I have been giving him a lecture," answered Gwen, seating herself on a low
chair by the writing table.

"What?" exclaimed Rachel. She could hardly believe she heard aright.

"I have been giving him a lecture," repeated Gwen. "Husbands occasionally
need one."

"My dear child what do you mean?" said her sister laying down her pen. "I had
hoped you might have been having a nice helpful talk with him."

"Well, I hope it has been helpful to him."

"You sound as if you had been rather impertinent," said Rachel not pleased.
"What have you been saying?"

"All husbands are alike," answered Gwen. "They get nice girls to marry them,
taking them away from their homes, and no sooner have they got them than
they seem to forget their existence. I have been studying husbands lately, that
is to say since my friend Mabel married. Men are fearfully selfish."

Rachel looked gravely at her sister.

"Gwen, I advise you to wait to give out your opinions till you are a little older.
You really talk like a very silly child. I hope if you have been saying anything
impertinent to Luke that you will apologise to him before you are an hour
older. I am quite horrified at you."

Rachel's face was flushed, and Gwen saw she was more angry with her than
she had ever been in her life. But she was not daunted. Here was her
favourite sister, whom she adored, tied for life to a man who was engrossed in
his parish and had no time whatever to think of her. She felt boiling with rage.

"I certainly shall not apologise," she said, "it would take away any little good
my words may have done. I think I have come to spy out the land none too
soon, and that Luke will awake to see that what I have said anyhow has some
sense in it, and that he will not let you carry the coal scuttle another time."

Rachel looking at Gwen's earnest and rather anxious face repented that she
had been so stern with her. After all she was only an ignorant child. She could
not expect an old head on young shoulders; besides, Gwen was always
putting her foot into it, talking of things about which she really knew nothing.
The family took her sayings for what they were worth and laughed at them.
She wished she had not taken her so seriously.

But the fact was, that Rachel was conscious that Luke sometimes surprised
her by not doing what he would have done during their courtship. He had
been very chivalrous in those days, and more careful of her than was
necessary. Now he often let her do things for him which he would in those
days have done for her. At times the consciousness of this had a little hurt her;
he seemed to have lost, where she was concerned, his old world courtesy.
She remembered feeling ashamed when Mrs. Stone had come to tea, that he
had let her, his wife, do all the waiting while he sat still and talked. He was so
interested in his conversation that he had never noticed it.

But these were such very little things after all, that Rachel had made up her
mind not to notice them. However, the fact that Gwen had noticed them made
her feel sore and somewhat indignant with her sister. But glancing again at the
child who had tears in her eyes at the thought that Rachel was wasted on
Luke, anger fled, and an amused smile took its place.

"Oh Gwen dear," she said, "I wish you could see how ridiculous you are. What
do all those little things matter when people love one another as Luke and I
love? You see you are too young to understand. I really advise you to put
away your silly imaginations." She ended up with a laugh.

"Well then," said Gwen, "I will give you advice, rather than Luke. Why don't
you teach him what to do?"

Rachel laughed out loud. "Don't be foolish," she said.

"I'm not foolish," said Gwen earnestly. "But I have read, and I think it is
probably true, that a woman can make a man what she wants him to be."
"Explain yourself," said Rachel amused.

"I mean that you should teach him to remember that as a husband he is
bound to follow your wishes. Tell him, for instance, to fetch the coals for you;
to open the door when you have your hands full; and to hand the tea about
when you have people. I have noticed that Mrs. Graham, who has one of the
best of husbands, does this, and the consequence is that he waits on her as if
he were her slave. You know, Rachel, at present you are Luke's slave."

"Well now you have done your lecture," said Rachel good-humouredly, "So
we'll go out, and I hope to hear no more of it; but I feel strongly you ought to
ask Luke's pardon for what must have struck him as great impertinence."

"I shall do no such thing," said Gwen. "I think you will find that he profits by my
words."

But as Luke took her home as usual at night, leaving her at the door of the
house in which was her room, having talked to her as if nothing had,
happened, Gwen felt rather small. It did not look much as if he had profitted or
indeed remembered her lecture. This was decidedly snubbing, but then Gwen
was used to being snubbed.

CHAPTER X.
THE TRAINING BEGINS.

Rachel was not very sorry to remember that Gwen would be leaving in a few
days. She might do a great deal of mischief if she stayed longer with them.
Anyhow she would probably make Luke unhappy if she talked to him in the
same way as she had spoken to her.

But there was only one part of Gwen's conversation that had effect on Luke,
and that was the fear of Rachel's health suffering from the change from the
country to the overpopulated town.

When he had left Gwen at the door of her lodging, he hurried home, and after
hanging up his hat in the hall, made his way to the drawing-room where he
knew he would find Rachel. She was working, but on his entrance looked up,
and their eyes meeting, both knew that Gwen was the subject of their
thoughts. Rachel was the first to speak.

"I don't know exactly what that silly child has been saying to you," she said,
"but I'm afraid she has been very impertinent."

"Well I can't deny that she has said some outrageous things," he said
laughing, "but after all she is only a child."

"And you must forgive her," said Rachel. "We never take any notice of what
Gwen says. She gets the most ridiculous notions into her head. I hope you are
not letting the thought of her worry you."

"A great deal of what she said was sheer nonsense," he answered, "but I own
what she hinted about your health distresses me. I only hope it is not true."

"My health? But what did she say? I am perfectly well."

"She has made me so anxious that I want you to go home with her for a few
weeks."

"Go home! And leave you behind! No thank you. It would do me no good at
all. Besides, I am perfectly well and don't need a change. What a stupid little
thing she is; but do look over her folly and try and like her," said Rachel. "She
has such good points. For instance, she is perfectly true."

"Possibly," said Luke, smiling; then he added, "I can't say I am exactly
enamoured of her."

"No, but when you know her better you will see her virtues. I am afraid she
has been really impertinent to you."

Luke did not answer. He leaned forward and looked at Rachel anxiously.
"Are you sure that you are feeling well? Gwen seems to think that you are
tired out. Is that the fact?"

"Tired out? What with? I have only this tiny house to see after; in fact I don't
think I have enough to do."

Luke sighed a sigh of relief.

"Then I needn't worry?"

"Certainly not. Put it right from your mind. It is only a child's nonsense."

And Luke did as he was asked and worried no more about her.

He left her to write some letters before going to bed, and Rachel sat working;
but her thoughts were busy.

Although Gwen had talked a great deal of nonsense was there not a grain of
truth in some of her words? "A woman can make a man what she wants him
to be," she had said: she had evidently read this in a book, it had not come out
of her own little head. Rachel supposed there was some truth in the words;
and possibly she had been unwise herself in not insisting more that the
attentions that had been shown her during her courtship should not be
dropped now that they were man and wife. She was afraid that she had
unwisely done things herself instead of asking Luke to do them, and then was
surprised that he had lost the habit of waiting upon her. She had got in the
way of waiting on him and of saving him all extra effort when he came in from
his work in the parish.

She knew that in Luke's case it was often simply absentmindedness that
prevented him seeing of what she was in need at the moment. Once buried in
a book nothing would arouse him save her voice; or if he was in the midst of
an argument with a fellow clergyman, he would quite unconsciously allow
Rachel to help them both to tea though it meant rising from her seat. At times
she had felt a little indignant at the two men sitting while she served them; but
on the other hand if at her request he handed round the hot tea cake, he
would stand with the plate in his hand talking, while the contents got cold, or
would absently hold the kettle while Rachel watched in anxiety lest the water
should pour out on to the carpet, or on to his foot. It was easier to do these
things herself. She had not known that anyone notice these little omissions on
Luke's part; but evidently Gwen had taken count of them at once.
"A woman can make a man what she wants him to be." Yes, but save in these
few insignificant matters Luke was exactly what she wanted him to be, and in
these small matters perhaps she had been at fault, not him. Gwen had
opened her eyes; though she would not tell her so. Rachel felt that she had
made it easy for him to neglect little home courtesies. When the child had
gone she would behave somewhat differently.

Gwen came to breakfast next morning just as if nothing had occurred between
her and her brother-in-law: and Luke, who had put away the thought of
Rachel's health being affected by living in Trowsby, was too large-minded to
bear any grudge to the girl for the audacious things she had said to him. He
banished them from his mind, recognising the fact that Gwen was after all only
a child, and would learn better by-and-bye.

Rachel, however, found her a little trying, as Gwen after breakfast, took her to
task about more than one matter.

"You should be the President of a Mutual Improvement Society, Gwen," she

said laughing. "You have got terribly into the habit of setting people to rights,
or rather trying to do so. You want to go through a course of snubbing, my
child. Have you apologised to Luke yet?"

"Certainly not. And you know Rachel I can't help thinking that my lecture has
done him good. When I came in I saw him actually pouring the water into the
tea pot for you."

Rachel laughed, but she did not inform Gwen that she had begun the training
of her husband that morning. And that Luke had risen to it as if it were a
matter of course. He was, in fact, perfectly unconscious that he had not
always poured the water from the kettle into the teapot for his wife.

"You see I was right after all," continued Gwen. "Men only want to be taught
what to do."

"You were a very impertinent little girl," said Rachel. "And Luke felt you to be
so, only he is too kind and noble to remind you of it this morning."

"Well I shall remind him of it later on," said Gwen calmly, "as I have a few
more home truths to tell him."

"I forbid you to do anything of the sort," said Rachel, really angry now. "You
have no idea how ashamed I am of you, nor how much harm you might have
done if Luke was not as good and kind as he is."
Gwen, who was helping to clear away the breakfast things, stood still with the
plates she was carrying and looked at Rachel.

That the sister to whom she was so devoted could possibly speak to her in
such a severe tone of voice when she had been doing all she could, as far as
she knew, to help her, went to her heart. She stood still and looked at her with
tears in her eyes.

"Are you really ashamed of me?" she asked with a catch in her voice.

"Yes I am. I can't think how you could possibly have spoken impertinently to
Luke."

Gwen gave a little sob.

"I didn't mean to be impertinent," she said, "It was only because I love you so
much and couldn't bear to find you in this horrid pokey little house and looking
ill and tired. I don't see why you should feel ashamed of me when it was all my
love that did it," and Gwen laid down the plates to find her handkerchief.

Rachel's tender heart relented.

"Don't cry Gwen dear," she said, putting a hand on her shoulder. "I know you
didn't mean to do any harm; and as a matter of fact I am sure no harm has
been done; but you must remember it does not do to talk over a wife with a
husband. It is not wise."

Gwen threw her arms round Rachel promising to ask Luke's pardon for
speaking as she did. She assured Rachel she would do anything in the world
for her. And she kept her promise. No sooner did she hear Luke open the front
door and go up to his study just before dinner, than she ran after him. She
was no coward.

"I expect I was impertinent to you yesterday," she said, looking at him straight
in the face, "at least Rachel tells me I was. I didn't mean to be; only I meant to
tell you the truth and you know husbands do sometimes need the truth to be
told them."

Luke laughed heartily.

"Happily," he said, "it was not the truth so it does not signify in the least. I
shan't think of it again."
"Oh but it was the truth," said Gwen flushing, "but I don't mean to say anything
more about it. I might perhaps have said it more gently and in a more polite
manner; and I'm awfully sorry that Rachel is ashamed of me."

Luke fancied he heard a little catch in her voice and looked at her kindly. He
could hardly refrain from laughing out loud at her.

"Well you can put it out of your head and not think of it any more. I quite
appreciate the fact that it was out of your love and anxiety for Rachel that you
spoke as you did, and so we will be good friends again."

Gwen looked down and her lips trembled. "Thank you," she said. "And you will
take care of her, won't you."

He patted her on the shoulder and told her to run away as he was busy. And
Gwen, having no more to say, obeyed; but she felt rather small.

She resented the pat on her shoulder just as if she were a child. She was not
sure that she liked Luke at all.

"I can't think," said Gwen after dinner, as she and her sister were sitting
working in the drawing-room, "how it is that you don't show in the least that
you do any house work. You look as dainty and as pretty as ever."

Rachel laughed.

"When I began to realise that a great part of my day would be taken up with
dusting," she answered, "I bought the prettiest overall I could find."

"You look as if you had just come straight out of the garden and ought to have
your hands full of roses." Gwen looked with adoring eyes at her sister, adding:

"But there is a new expression on your face somehow. I think you are really
prettier than ever."

"If so it is love that has made me so," said Rachel.

Gwen laughed. "I don't quite believe that," she said.

Rachel smiled to herself, as she thought of the day on which she had bought
the overall, and had shown it to her mother-in-law with pride.

Mrs. Greville had looked at it critically, remarking:

"But you need not have gone to the expense of getting such a fanciful thing.
You could have got a yard or two of some good strong material and made it up
yourself. It would have served your purpose quite as well."

"I don't think so," Rachel answered laughing, "you see I like to be ornamental
as well as useful."

"You need not worry about making yourself ornamental," said Mrs. Greville.
"What you really want to do is to strive to be useful."

"Oh mayn't I be both? I do believe in beauty. I think our houses and everything
we possess should be made as beautiful as possible. It makes life easier and
happier." Unconsciously she looked round at the drab walls and ugly furniture.

Rachel would not on any account have complained of either to her mother-in-
law; and her glance round had not been meant to imply anything of the sort to
her. It had been done before she realised what she was doing or how her look
might be interpreted. But by the sudden change in Mrs. Greville's expression
of face she recognised what a mistake she had made.

Mrs. Greville had put a great restraint on herself ever since Rachel's arrival,
and had been most careful not to show her disappointment in Luke's choice of
a wife, to her daughter-in-law.

But Rachel's unappreciative glance round at the walls and furniture hurt her
inexpressibly, as she had lain awake many nights planning how she could
make the little house as homelike and attractive as possible. She quite
thought she had succeeded. Having lived all her life with early Victorian
furniture she saw nothing ugly in it; and indeed it struck her as both homelike
and comfortable. She had, moreover, spared several pieces of furniture which
she had decidedly missed when she had had to turn out into a barely
furnished room for the sake of her son's wife. But evidently nothing that she
had done for Rachel's comfort was appreciated. The disappointment was so
great that she turned a little pale.

"I am sorry Rachel," she said, in a strained tone of voice, "that we were not
able to supply you with Sheraton furniture. You see you have changed a
luxurious home for a poor one and must bear the consequences. We have to
cut our coat according to our cloth. I am sorry that our efforts are so painful to
you."

Rachel had flushed crimson.

She was tongue-tied for the moment. She could not tell a lie and say that the
furniture, which she had labelled in her mind as hideous, was to her taste. She
looked beseechingly at Mrs. Greville.

"I am sorry, my dear, if I have distressed you by my remark," said Mrs.

Greville, "but don't try to explain the look you gave at the furniture, I could not
possibly mistake its meaning."

Then while Rachel in her confusion and distress murmured her regret, Mrs.
Greville looked round the dining-room.

"I think perhaps I ought to have had the walls papered afresh and a lighter
colour," she had said. "I daresay it looks a little dull to a young creature like
you, and," she added, remorse getting the better of her, "I ought to be grateful
to you, for though you don't like my papers you love my son," and Mrs.
Greville ended by bestowing a hearty kiss on her son's wife before hurrying
away.

Rachel was left standing in the middle of the room with her eyes full of tears.
Something about her mother-in-law had touched her for the first time; and she
began to wonder if she might not possibly in the future learn to love her. She
wondered too how she could ever look her in the face again. She must have
seemed so terribly ungrateful and ungracious, not to say ill mannered. But her
glance round the room at the walls and its furniture had been quite involuntary,
and she had had no intention whatever of letting Mrs. Greville know how she
disliked them.

She smiled now as she remembered her mother-in-law's criticism on her

pretty overall, but the smile faded as she realised that though she had taken
the Bishop's advice and was trying hard not to allow her thoughts to rest on
the trials that she had had to meet in her new home, she had not by any
means yet succeeded in learning to love her mother-in-law.
 CHAPTER XI.
THE CHOIR THREATEN TO STRIKE.

It had not taken Rachel long to discover that she had married an untidy man.
Being very tidy and dainty in her ways herself, this discovery was rather a
shock to her. But she came to the conclusion that Luke's mind was so full of
the things that really mattered, that the less important things were nothing to
him though they meant a great deal to her.

Remembering her promise, the morning after she had prepared his study for
him, she had awakened earlier than usual and had gone there the first thing.
She had promised him that she would be responsible for keeping it in order
and that Polly should have nothing to do with it. On opening the door she
stood still and laughed at what she saw.

It looked to her in terrible disorder! Though he tried to convince her afterwards

that there was such a thing as a tidy untidyness. He knew just where
everything was, he said, and could lay his hand upon it.

But to Rachel's eyes disorder reigned.

Because the waste paper basket was not just at hand, he had thrown on to
the floor his many torn up letters. Books were piled on the ground. His table
was strewn with papers: there was scarcely a chair without some volume of
reference on its seat.

Rachel picked up all the torn letters putting them into the waste paper basket,
arranged the books, with a certain amount of trepidation, on the shelves, and
finally lifted up every letter and paper from the table to dust them, laying them
back in exactly the same position as she found them. She opened the window
carefully, anxious lest some of the papers that strewed the desk should take to
themselves wings and fly away.

It amused her the first morning, she felt that Luke had indeed needed a study.
But the care of the room added to her work and took time as she had to be so
careful not to disturb anything. And in her heart of hearts she wished that Luke
was tidy!
Another thing that troubled her was the fact that on the muddiest days Luke
would run upstairs without wiping his boots. He was always in such a
desperate hurry to get through the numberless letters that awaited his
attention on his study writing table that he would hurriedly hang up his hat in
the hall, and then spring upstairs two steps at a time and shut himself in. The
necessity of rubbing his boots never occurred to him. His mind was full of
important matters, things that had just taken place, and letters that had to be
posted. Then to brush his coat and hat before going out never crossed his
mind. He was always so hurried, and Rachel supposed that he expected to
find everything that was necessary to be done, done for him.

When the snow came in December she remonstrated with him once about his
boots, and on looking at the marks of his footsteps on the stairs he was filled
with remorse, as he recognised that he had made unnecessary work for
Rachel.

But he forgot it next time, and his wife felt that to remind him again and again
would only worry and fret him. For she recognised that the work he had to get
through was immense and that it was her duty to make life as easy for him as
possible. He worked far too hard, and it seemed to Rachel that the time he
could spend in his own home grew shorter every month. She was getting used
to it, and though at times the winter evenings felt long to her, and it was
somewhat of an effort not to give way to low spirits, she fought bravely against
melancholy, and always had a smile of welcome for her husband.

She had made a few friends by the winter and now and then Mrs. Stone would
run in with her knitting to spend the evening with her.

Rachel was conscious that Luke by his outspokenness made enemies and
that all was not harmonious in the parish, so it was a comfort to know that in
Mrs. Stone, both she and her husband had a valiant supporter, and that she
would act as peacemaker whenever she had the chance.

It must be confessed, however, that Mrs. Stone liked the role she had
undertaken for she was very fond of giving advice. She had taken a fancy to
Rachel and pitied her. Mrs. Greville, senior, was no favourite of hers, and
though Rachel was far too loyal to talk over her husband's mother with any
parishioner, Mrs. Stone could not but gather sometimes from silences on
Rachel's part, and by what she saw and heard from others, that Mrs. Greville
was deeply disappointed in her son's wife; and naturally a sensitive girl like
Rachel must be aware of the fact. It was to Mrs. Stone's credit that she kept
her views to herself and discussed no-one belonging to the Vicar with his
parishioners.