You are on page 1of 7

PLEASE NOTE

The paper “Evolutionary Techniques for Fault Tolerance” and its journal version “Evolving Inherently Fault-Tolerant
Systems” contain a small technical error. The text defines a measure
 
ê = e evolved fitness  100%
whereas the correct definition is  
ê = e f d  100%
The graphs, and all of the discussion and conclusions, are unaffected; only the original definition of ê in the text is in
error.

Adrian Thompson, 1998.


Proc. UKACC Int. Conf. on Control 1996 (CONTROL’96), pp693–698, IEE Conference Publication No. 427.

EVOLUTIONARY TECHNIQUES FOR FAULT TOLERANCE

Adrian Thompson

University of Sussex, UK

Abstract: Artificial evolution can integrate fault toler- RAM chip are directly encoded bit-for-bit onto a linear
ance considerations into the automatic design process, bit-string genotype, along with binary representations of
producing inherently fault-tolerant designs without ex- various parameters pertaining to timing and synchronisa-
plicit redundant parts. Population dynamics can give rise tion. The system has some interesting properties which
to some level of fault tolerance ‘for free.’ Requirements are not directly relevant here — see [11] for full details.
for fault tolerance can also be incorporated into the fit- The robot’s task is simply to move about while avoiding
ness function. The practicalities of these methods are in- walls, and all fitness evaluations take place with the real
vestigated, grounded in the study of a real-world evolved electronic control system (configured according to the
electronic control system for a robot. individual’s genotype) controlling the real motors. Stan-
dard genetic algorithm techniques can produce a config-
1. INTRODUCTION
uration for this hardware controller that induces excel-
This paper builds on earlier work [9] to show how ar- lent real-world wall-avoidance behaviour in the robot.
tificial evolution — such as a Genetic Algorithm (GA) See Goldberg [2] for the standard techniques and terms
— can automatically design systems that are inherently used in this paper. Simple though it is, this robotic exam-
tolerant to faults. In a harsh environment or a safety crit- ple will ground the following discussions in a real-word
ical application, a system might be required to retain a application.
certain level of ability even if a computer’s memory be-
In the next section, a detailed investigation is presented
comes slightly corrupted, or a few transistors fail. Toler-
to show how evolutionary population dynamics can en-
ance to semiconductor defects increases both yield and
dow an evolved system with some degree of fault tol-
feasible chip size, and is a necessity for wafer-scale inte-
erance automatically. In section 3, the practicalities of
gration. Evolution can integrate into the design itself an
explicitly specifying fault tolerance requirements in the
ability to function in the presence of faults, rather than
fitness function are discussed. Finally, we note that evo-
relying on the use of spare parts as is conventional.
lution can build working systems from faulty parts. This
It is important that evolution operate at the same level of toolbox of evolutionary methods can be combined with
abstraction as the faults to be tolerated manifest them- traditional redundancy techniques, and allows fault toler-
selves: it would be a mistake to evolve a neural network ance considerations to be integrated with the (automatic)
to tolerate perturbations to its structure, simulate it on a process of design.
digital computer, and expect the system to cope with fail-
2. GRACEFUL DEGRADATION FOR FREE
ures of the computer — the simulation program would
just crash. It has been observed in the study of molecular evolu-
tion that evolution tends to produce individuals which
Since tolerance to hardware failures is of particular inter-
not only have high fitness, but are also of a structure
est, we will draw an example from the field of Hardware
such that the average decrease in fitness caused by ge-
Evolution, where evolution operates at the level of elec-
netic mutations is small [1, 4]. In [9], it was noted that if
tronic design. Recent advances in Field Programmable
some of the possible genetic mutations cause exactly the
Gate Array (FPGA) technology mean that artificial evo-
same changes to the phenotype system as a particular set
lution can be allowed to manipulate reconfigurable hard-
of faults, then insensitivity to mutations gives insensitiv-
ware directly, without the need for simulation. The pos-
ity to those faults. Notice that it is not sufficient for muta-
sibility of evaluating evolving circuits physically instan-
tions to cause the same change in behaviour as the faults;
tiated in real silicon opens new possibilities for the ef-
they must actually make the same change in the underly-
ficient exploitation of semiconductor properties, and the
ing implementation. If this is the case, then some degree
interested reader is referred to [10]. Our example will be
of graceful degradation over that set of faults can arise
simpler, and is a RAM-based evolvable electronic con-
automatically from the nature of the evolutionary pro-
trol system for an autonomous mobile robot.
cess, without any special measures being taken to cause
The robot has a pair of wheels driven by d.c. motors, and this.
a pair of time-of-flight sonars. The control system archi-
To gain an rough understanding of the effect, consider a
tecture is approximately a RAM-based implementation
single individual in the population. The spread of this in-
of a finite-state machine. The 32 bits contained in the
dividual’s genetic information through the population in
successive generations depends not only on how many mean fitness of its bits. The fitness of a bit is determined
offspring it produces, but also on how many offspring by its own value (0 or 1) and the values of K other bits
those offspring produce, and so on. Each time one of (0  K  N-1).
these offspring is produced, however, it is subject to ge-
To generate a random landscape for particular values of
netic mutation. Hence an fit individual that is relatively
N and K, one proceeds as follows. For each of the N bits
insensitive to mutations will have mutated offspring that
in the genotype in turn, choose at random K other bits
are also fit; its genetic information can spread through
which will influence its fitness. Since the fitness of each
the population more readily than that of an individual
bit will be determined by its own value and that of its
which is equally fit, but which is vulnerable to mutations
K influencers, a bit’s fitness can be given by a look-up
table of 2K+1 real-valued entries. For each of the N bits,
and so produces mutant offspring of much lower aver-
age fitness. Such mutation-insensitive individuals there-
a separate fitness look-up table is randomly generated,
fore tend to displace more mutationally-brittle individu-
with entries uniformly randomly drawn from the inter-
als over time, even if the brittle individuals have slightly
val [0.0, 1.0]. This random choice of influencers and
higher fitness (as demonstrated in [1]).1
look-up table entries is now held constant, and defines
This section is an overview of recent work to discover if a particular fitness landscape which can be used in an
this effect can occur in the context of engineering — and evolutionary experiment.
if so, whether it is of significant magnitude. The answer
Low values of K give, on average, ‘smooth’ random
to both questions turns out to be ‘Yes,’ but not for all
landscapes: a genetic mutation to a bit will not hugely
types of GA.
alter the fitness of the genotype, because that bit influ-
First, an example. Recall that the central part of the robot ences the fitness contributions of few other bits. In the
controller described earlier is a RAM chip, and that the limit of K=0, there is a single global fitness optimum
32 bits held in the RAM are directly encoded bit-by-bit with no other local optima. Conversely, for high values
onto a region of the linear bit-string genotype. A ge- of K a genetic mutation is likely to have a large effect
netic mutation to this region has the effect of inverting on fitness (‘rugged’ landscapes). In the limit of K=N-1,
one of the bits in the RAM. This is exactly the effect a single mutation changes the fitness of a genotype to a
of an adverse single-stuck-at (SSA) fault in the memory value which is completely uncorrelated with the unmu-
array of the RAM. An SSA fault in the memory array tated fitness. For 0 < K < N-1, any particular random
causes a particular bit to read either always 0 (SSA-0) landscape is likely to have some regions which are more
or always 1 (SSA-1). By ‘adverse SSA fault’ we mean rugged than others, and the value of K determines on av-
an SSA fault causing a bit to read the opposite of what it erage how rugged it is overall.
should: its effect in inverting the bit is the same as that of
The experimental method was as follows. With N=20,
a genetic mutation to the locus (bit-position) of the geno-
a random landscape was generated for a particular value
type representing it. Here, then, is a situation in which a
of K. Starting from a randomly generated population of
tendency for evolution to produce individuals insensitive
100 genotypes, a generational GA was run for 1000 gen-
to genetic mutations translates into a tendency to pro-
erations with a particular selection scheme, mutation rate
duce RAM-based control systems that display graceful
and single-point crossover probability. At the end of this
degradation in the presence of adverse SSA faults in the
run, the fittest individual in the population was taken, and
memory array.
a check was made that it was at a local optimum with re-
Another example would be if the connection matrix of a spect to single mutations. If not, then the GA was started
neural network were directly encoded onto the genotype: again with a new random population on a new random
the effect would give heightened tolerance to spurious landscape until the final fittest individual was a local op-
creation and deletion of connections. It may be possible timum. We now wish to answer the questions, “Is this
deliberately to design the genetic operators and encoding evolved optimal individual less sensitive to single muta-
in engineering applications such that improved tolerance tions than one would statistically expect for a local opti-
to a pre-specified (rather than accidental) class of faults mum of this fitness to be, given the current landscape? If
results. so, by how much?”
The ‘fitness landscape’ of an evolutionary problem is the Taking the evolved optimal individual, the mean fitness
assignment of fitness values over the space of all pos- decrease f d caused by a single mutation was measured,
sible genotypes. For ease of experimentation, we shall averaged over all N possible single mutations. The al-
study evolution on the well-known NK model of fitness gorithm given in the appendix was then applied, to ran-
landscapes [5] rather than on a fitness landscape arising domly assign new fitnesses to the single-mutation neigh-
from a real problem. N is the length, in bits, of the geno- bours of the optimum, but such that (a) local-optimality
type. In this model, the fitness of each bit can be cal- is preserved and (b) the statistical correlation between
culated, and the fitness of the whole genotype is just the the fitness of the optimum and the fitnesses of its single-
mutation neighbours is preserved. If we now re-measure
the mean fitness decrease f d caused by single mutations
1 Here we assumed that the task was not changing over time. If

the task is rapidly changing, then [4] has demonstrated that quite the
opposite outcome can result. to the optimum, it will (on average) be typical of an op-
timum of this fitness on a landscape of the current N, K 12.0
and choice of influencers. The difference in mutation- e (%)
sensitivity between the optimum found through evolu- 10.0
tion and this random typical optimum of the same fitness
is e = f d f d . 8.0
For each particular setting of K and the GA parameters,
the entire procedure of the previous two paragraphs was 6.0
repeated at least 200 times2 and the values of e aver-
aged to give e. The value e gives the expected differ- 4.0
ence between the fitness drop when a single mutation is
applied to an evolved optimal individual and the fitness 2.0
drop that one would statistically expect on optima of the
same fitness under the same conditions. Below, we will 0.0
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
express e as a percentage ê of the mean fitness of the mutation rate (expected bits/string)
final optimal solutions found by the GA (averaged over
all the runs). So where a fitness drop of k% would nor- 0.75
mally be expected on optima of a particular fitness when Fitness
0.74
a single mutation is applied, if such optima are found
through evolution then the actual fitness drop will only 0.73
be (k ê)% on average. ê is the percentage of unmu-
tated fitness by which the single-mutants of an evolved 0.72
optimum are better than one would expect, on average.
0.71
This experiment has been performed for over a hundred
combinations of K, mutation rate, crossover probability 0.70
and selection method. Figure 1 shows the results for
K=10, crossover probability=1.0, with linear rank selec- 0.69
tion. ê increases with the mutation rate until the ‘error
0.68
threshold’ is reached: beyond this the mutation rate is 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
too high for the GA to work properly and both ê and mutation rate (expected bits/string)
the actual fitness attained decrease. Very similar results
are obtained when the selection method is fitness pro- Figure 1: ê (top) and mean fitness of evolved optima
portional with linear scaling. It seems to be generally (bottom) as the bitwise mutation probability is varied.
true that êmax occurs at the maximum mutation rate for K=10, crossover probability=1.0, linear rank selection.
which the GA still works well (before the fitness starts to The error bars indicate +/- the standard error.
decrease due to the ‘error catastrophe’). Fortunately, this
maximum rate of mutation — which depends on the fit-
ness landscape and the selection pressure — is also the was used, ê was at least as great as for the other selec-
mutation rate which would normally be used for opti- tion methods, and was maximised at the largest value of
mum performance. The maximum ê observed under any T that could be used without the fitness suffering. Un-
conditions is that seen in this figure: 10.6%. der the particular conditions used, ê was maximised at
T =60%. As T was reduced to 5%, ê fell to around a
For low or high K (K = 0, 5, 15, 19 have also been fully tenth of its maximum value even though the fitness ob-
explored) the maximum value of ê is smaller: the ef- tained was unaffected.
fect occurs most on landscapes of intermediate smooth-
ness/ruggedness. As the crossover probability is reduced In conclusion, for landscapes of intermediate smooth-
from 1.0, êmax is also reduced, with the maximum value ness/ruggedness, and when elitism is not used, evolved
of ê without crossover being about half of that when the optima have been observed to be around 10% less de-
crossover probability is 1.0. If elitism was introduced graded by single mutations than would be statistically
into the rank selection method (i.e. the fittest individual expected for that problem. The GA parameters did not
is carried over into the next generation without mutation have to be set in an unusual way to achieve this. If the
or crossover), then although the fitness obtained by the genetic encoding is such that some genetic mutations
GA was greatly improved, êmax was reduced to around make the same change to the phenotype as do a particu-
quarter of what it would otherwise be. lar class of faults, then this effect translates directly into
tolerance/graceful degradation in the presence of faults
In truncation selection with threshold T , the T % best in- of that class. The effect can be used to augment other
dividuals have equal probability of reproducing, and the means of fault tolerance in engineering applications be-
others have zero probability. When truncation selection cause it arises ‘for free.’ The effect can certainly occur
2 Often as many as 1000 runs were performed, as deemed necessary in realistic applications, but its magnitude outside of the
by monitoring the standard error of the final mean value. theoretical NK model is not yet known.
3. EXPLICITLY SPECIFYING for by this consensus sequence was then tested in the
FAULT-TOLERANCE REQUIREMENTS presence of each of the 32 possible adverse SSA faults
in turn. The fault that caused the consensus individual to
behave the most poorly (lowest fitness score) was nom-
The obvious way to induce evolution to produce a fault-
inated as the ‘current fault.’ Another generation of evo-
tolerant design is to incorporate a fault-tolerance mea-
lution was then performed, but with the current fault be-
sure into the fitness function. That way, fault tolerance is
ing present during all of the fitness evaluations. After
explicitly part of the required behaviour. Ideally, for each
this generation the new consensus individual was con-
fitness evaluation the individual would be given a trial in
structed, tested, and a (possibly) new current fault nom-
the presence of every possible fault in turn, and the re-
inated for the next generation. The process continued
sulting fitness score would be some measure of perfor-
in this way, with a single fault being present throughout
mance in the face any fault. For systems being evolved
all evaluations within a generation — this fault being the
in software simulation, it is easy to simulate the effects
one that caused the worst performance in the consensus
of faults. If the individuals are instantiated in reconfig-
individual of the previous generation.4
urable hardware for their fitness evaluations (see the in-
troduction), then many faults can be emulated simply by Figure 2 shows that the maximum and mean fitnesses
altering the configuration from what it would normally dropped sharply at generation 85 when faults were first
be. For instance, in our robot controller example, an ad- introduced, but over the course of the next 150 genera-
verse SSA fault in the RAM chip’s memory array can be tions returned to high values. Figure 3 shows that when
emulated by writing the wrong value to that bit. the faults were first applied the controller was already
tolerant to most SSA faults, but that a few were criti-
To have each evaluation consist of trials for every pos-
cal. At various stages afterwards, this tolerance to most
sible fault — of which there are typically many — will
SSA faults is lost in the GA’s attempts to improve per-
normally be prohibitively time consuming. However, if
formance on the single most serious current fault. Some
we are interested in optimising worst-case performance
serious faults are seen to persist over long periods. Even-
(i.e. minimising the effects of the most serious fault),
tually, consensus individuals arose that give satisfactory
there is a potential short-cut. In this case the fitness
performance when any of the SSA faults is present.5 Fig-
measure will be based on performance in the presence
ure 4 compares the fault tolerance of the conventionally-
of only the single most serious fault. If some way of
evolved consensus individual at generation 85 with that
predicting which fault is the most serious can be found,
of the first completely-tolerant consensus which arises
then only this single fault needs to be introduced during
at generation 204. The criterion for ‘satisfactory per-
the fitness evaluation. A similar situation arises if only
formance’ was for the real robot to display what would
a relatively small subset of the possible faults seriously
reasonably be called wall-avoiding behaviour, and corre-
sponds to a fitness score of  1:0.
degrades the system: we need only consider the small
subset of serious faults.
Returning to the general discussion, we can see that this
However, which faults are the most serious might be dif-
example has exploited the similarity between individu-
ferent for each individual in the population. If the only
als in the population by predicting that a single fault will
way to identify the worst faults for each individual is to
be the most serious one for all individuals at a particu-
test them with each fault in turn, then we are back where
lar generation. This fault was identified by exhaustively
we started. In practice, though, after the first few gener-
testing a single ‘average’ individual — the consensus.
ations the individuals are mostly similar and the popula-
Though this fault-prediction strategy is not exact, it had
tion as a whole changes gradually over time. These facts
the desired effect of catalysing the evolution of a com-
can be used in predicting which faults are the most se-
pletely fault-tolerant individual.
rious without having to test every individual with every
fault; fortunately small errors of prediction are unlikely Many other strategies could be used to decide which
to be disastrous to the evolutionary process. faults an individual should encounter during its evalu-
ation: the example above is just intended as a simple il-
To illustrate this idea, we evolve the RAM-based robot
lustration. If there were a very large number of possible
controller example to give satisfactory wall-avoidance
faults, exhaustive testing of even just the single consen-
behaviour in the presence of any of the 32 possible ad-
sus individual per generation could take too long. Fol-
verse SSA faults in its RAM chip. First, the wall-avoider
lowing a suggestion made in [9], an attempt was made
was evolved as normal, using a standard generational
to co-evolve [3] a population of faults — the idea be-
GA with rank selection, elitism3, and a population size
4 It would have been better to take the consensus of the current
of 50. After 85 generations the GA had stabilised at a
good solution. Then the consensus sequence was gen- generation rather than of the previous one: this was an error. The
fact that the population changes relatively little between consecutive
erated: the genotype formed by, for each locus, taking
whichever of the values f0, 1g was most common in the
generations, although not deliberately exploited in this example fault-
prediction strategy, means that this mistake was not ruinous.
5 In fact, if the GA was left to run, then these completely-tolerant
population at that position. The robot controller coded
solutions would be lost again as the GA concentrated entirely on im-
3 The use of elitism means that the effect identified in the previous proving performance in the presence of the current most serious fault
section will be small. — even if that performance was already satisfactory.
The 32
adverse
SSA
faults.

85 135 185 235


Generations
Figure 3: The evolution of fault tolerance: results of the exhaustive test over all possible adverse SSA faults made on the
consensus individual of each generation. The darker a spot, the more serious the fault. Pure white represents satisfactory
performance (fitness  1:0), and pure black the worst possible performance. At the generations marked with arrows, the
consensus individual is satisfactory in the presence of any SSA fault.

Fitness Fitness
1.6
1.60 After
1.4
1.40
1.2
1.20 Satisfactory performance
1.0
1.00
0.8
0.80
0.6
0.60 0.4
0.40 Before
0.2
0.00 100.00 200.00
0.0
Generations

Figure 2: Maximum and mean fitness in the population 32 possible adverse SSA faults
over time. The first 85 generations were in the absence of
faults, thereafter all fitness evaluations were in the pres- Figure 4: Fault tolerance of the consensus at generation
ence of the ‘current fault’ (see text). 85, and then after 119 generations of evolution in the
presence of faults. In each case, the faults have been
sorted in order of severity.
ing that evolution could be used to maintain a popula-
tion of faults that concentrates on the weak-spots of the
co-evolving target population and tracks them over time.
Unfortunately, there was not enough correlation between
the positions of the most serious faults for evolution to
identify them more efficiently than random search, and
the experiment failed. This may be a general difficulty
for such techniques, but more investigation is needed. distributed GA [7].
Another interesting possibility is the use of a steady-state There is clearly more work to be done. However, what
(rather than generational) GA [8]. Here, for a successful has been shown here is that if some way of targeting the
individual to stay in the population, it must score well most serious weak-spots of individuals can be found,
in repeated re-evaluations which could be used to grad- then subjecting the individuals to these faults during
ually build up an accurate picture of performance in the their fitness evaluations can cause the evolution of sys-
presence of a set of faults. The difficulty here is that if tems tolerant to all of the possible faults. This has been
relatively few out of a large set of faults are serious, then demonstrated in evolving fault-tolerance in a real-world
the population can be dominated by new individuals that robot controller. It may be possible to use an adaptive
have been lucky enough not yet to have encountered the process such as co-evolution to target the weak-spots,
faults which affect them. A further embellishment that or search using application-specific heuristics may prove
has proven useful in a similar problem is the use of a more appropriate.
4. FAULT RECOVERY APPENDIX
The NK-landscape altering algorithm (section 2).
If evolution is allowed to continue as a corrective mech-
anism after a persistent fault has appeared, then it will Taking the locally optimal genotype g found through
change the design to compensate for the new behaviour evolution:
of the faulty part. In fact, evolution does not ‘know’ REPEAT f
what behaviour the components are supposed to exhibit, FOR each possible single mutation m f
so will exploit whatever properties they do possess, even FOR each locus l f
if faulty. [9] described an experiment where one of the Would mutation m change which location in
adverse SSA faults giving unsatisfactory performance l’s fitness lookup table was accessed? If so,
in the robot controller example was permanently intro- then set the entry at this new location to a new
duced, and then evolution allowed to continue. At first, random value in the interval [0:0; 1:0].
g
the fitness of the population was dramatically lowered,
g
with none of the individuals performing as well as the
best of the population used to, but after only 10 gener-
g UNTIL this process has allocated new fitnesses
such that g is a local optimum.
ations the mean and best fitnesses of the population had
returned to their previous values. A population has more
REFERENCES
tolerance to newly occurring faults than any single indi-
vidual, because it already contains a diversity of slightly [1] M. Eigen, 1987. New concepts for dealing with the
different solutions. evolution of nucleic acids. In “Cold Spring Harbor
Symposia on Quantitative Biology,” vol LII.
In some applications, it may be possible to have evo-
[2] D. E. Goldberg, 1989. “Genetic Algorithms in
lution permanently running ‘in the background,’ to auto-
Search, Optimisation & Machine Learning.” Ad-
matically cope with changing component characteristics.
dison Wesley.
This applies to gradual drift in component properties as [3] W. D. Hillis, 1992. Co-evolving parasites improve
much as to faults, which are just an extreme case. In- simulated evolution as an optimization procedure.
deed, the other techniques described in this paper may In Langton et al., eds, “Artificial Life II,” pp313–
also be useful in evolving systems that take account of 324. Addison-Wesley.
component variations (eg. ‘engineering tolerances’). [4] M. Huynen & P. Hogeweg, 1994. Pattern gener-
5. CONCLUSION ation in molecular evolution: Exploitation of the
variation in RNA landscapes. J Mol Evol,39:71–79
Evolution can produce systems which by the nature of [5] S. A. Kauffman, 1993. “The Origins of Order.”
their design exhibit fault tolerance or graceful degrada- Oxford University Press.
tion. Conventional design methodologies cannot cope [6] D. Mange et al., 1996. Embryonics: A new fam-
with integrating fault tolerance requirements into the ily of coarse-grained field-programmable gate ar-
heart of the design process, and must resort to provid- ray with self-repair and self-reproducing proper-
ing spare parts (redundancy). It is also possible to use ties. In Sanchez and Tomassini, eds, “Towards
evolution to produce fault-tolerant designs augmenting Evolvable Hardware: The evolutionary engineer-
redundancy built into an underlying implementation: [6] ing approach,” pp197–220. Springer LNCS 1062.
describes a suitable evolvable VLSI architecture with [7] M. McIlhagga, P. Husbands, and R. Ives, 1996.
redundancy and built-in self-test. Three evolutionary A comparison of search techniques on a wing-box
techniques have been described: some degree of fault- optimisation problem. In “Proc. Parallel Problem
tolerance can arise ‘for free’ from the nature of the evo- Solving from Nature (PPSN) IV.” (to appear)
lutionary process; fault tolerance requirements can be [8] J. Paredis, 1994. Steps towards co-evolutionary
explicitly written into the fitness function, and this can classification neural networks. In Brooks and
be made efficient by actively targeting the weak-spots Maes, eds, “Proc. Artificial Life IV,” pp102–108.
of the evolving population; finally, evolution can be al- MIT Press.
lowed to build a working system from faulty parts. Ar- [9] A. Thompson, 1995. Evolving fault tolerant sys-
tificial evolution holds the promise of integrating fault tems. In “Proc. of the 1st IEE/IEEE Int. Conf.
tolerance and automatic design. on Genetic Algorithms in Engineering Systems
(GALESIA’95),” IEE Conf. Pub. No. 414, pp 524–
ACKNOWLEDGEMENTS 529.
This work is supported by a D.Phil. scholarship from [10] A. Thompson, 1996. Silicon evolution. In “Proc. of
the School of Cognitive & Computing Sciences. Spe- Genetic Programming 1996 (GP96).” MIT Press.
cial thanks to Phil Husbands, Dave Cliff, Inman Harvey (to appear)
and Giles Mayley. [11] A. Thompson, I. Harvey, and P. Husbands, 1996.
Unconstrained evolution and hard consequences.
In Sanchez and Tomassini, eds, “Towards Evolv-
able Hardware: The evolutionary engineering ap-
proach,” pp136–165. Springer LNCS 1062.

You might also like