Evolutionary Learning

Chapter 12

Introduction
it models the genetic process that gives rise to evolution it models the sexual reproduction where both parents give some genetic information to their offspring it works very well but the solution is not guaranteed it is very popular algorithm for people to use when they have no idea of any other way to find a reasonable solution. it perform both exploitation and exploration so that they can make incremental improvements to current good solutions, but also find radically new solutions, some of which may be better than the current best

Genetic Algorithm
GA is a computational approximation to how evolution performs search, which is by altering the genome and this changing the fitness of individuals In order to interpret the theory into algorithm we need to figure out
a method for representing problems as chromosomes a way to calculate the fitness of a solution a selection method to choose parents a way to generate offspring by breeding the parents

To describe above methods we will use NP-complete problem example known as Knapsack problem in following slides

Knapsack problem
Suppose that you are packing for your holidays.
You've bought the biggest and best rucksack that was for sale, but there is still no way that you are going to fit in everything you want to take (camera, money, addresses of friends, etc.) and the things that your mum is insisting you take (spare underwear, phrasebook, stamps to write home with, etc.). As a good computer scientist you decide to assign a value to each item, and measure how much space it takes up.

Then you want to maximize the value of the items you will take with you, with the constraint that everything has to fit into the bag.

String Representation
way to represent the individual solution, in analogy to the chromosome these strings are composed of some chosen alphabet and most of the time alphabets are binary each alphabets of the string are analogous to alleles we create a set of random strings to be our initial population

String Representation in Knapsack Problem

in this case we make binary string consider string of length L, where L is the total number of things we would like to take with us, and make each unit a binary digit we then encode a solution using 0 for the things we will not take and 1 for the things we will.

if there were four things we want to take, then (0, 1, 1, 0) would mean that we take the middle two, but not the first or last

Evaluating Fitness
A fitness function is a particular type of objective function that is used to summarize, as a single figure of merit, how close a given design solution is to achieving the set aims.

Fitness function can be seen as an oracle that takes a string as an argument and returns a value for that string
It is problem-specific part of the algorithm.

Clearly, best string should have the highest fitness value and the fitness value should decrease as the strings do less well on the problem.
Population quality depends upon the fitness value they have.

Evaluating Fitness in Knapsack Problem

For knapsack problem, we could decide that we want to make the bag as full as possible. So we would need to know the volume of each item that we want to put into the knapsack, and then for given string that says which things should be taken, and which should not, we can compute the total volume. This is then a possible fitness function.

However, fitness function does not tell us anything about whether they will fit into the bagwith this fitness function the optimal solution is to take everything.

Evaluating Fitness in Knapsack Problem (contd..)

We again check if the string with higher fitness value fits in the sack. If it fits it is still a fit solution if not we decrease the value of fitness. We calculate the fitness value as the sum of values of the items to be taken if they fit into the knapsack, but if they do not we will subtract twice the amount by which they are too big for the knapsack from the size of knapsack.

Population
Usually, first population in GA is created randomly After fitness of every string in evaluated, first generation is bred together to make second generation, which is then used to generate third, and so on. After the initial population is chosen randomly, the algorithm evolves is such a way that the fitness of individuals in the population increases over the generations.

Population in Knapsack Problem

For this problem, we will now create a set of random binary strings of length L by using the random number generator. Fix number (lets say 100) of strings are selected as total population of first generation. Then, we choose parents out of this population, and start breeding them.

Unlike the real evolution, at every iteration in this problem, population stays the same size i.e. 100 .

Parent Selection
We need some method to select best possible parents in order to improve the fitness of new generation idea here is that fitness will improve if we select strings that are already relatively fit compared to the other members of the population (exploitation) however, it is also good to allow some exploration in there, which means that we have to allow some possibility of weak strings being considered Basic idea of exploration is to choose strings proportionally to their fitness, so that fitter strings are more likely to be chosen to enter the mating pool.

Parent Selection Methods

1. Functional Selection
Simple fractional method

2. Fitness Proportional Selection

Probabilistic model

Functional Selection
Comparatively, it is simple method Just pick up some fraction f of the best strings and ignore the rest It is very easy to implement, but it does limit the amount of exploration that is done, biasing the GA towards exploitation

Fitness Proportional Selection

Strings are selected probabilistically, with the probability of a string being selected being proportional to its fitness. Function generally used is (for string )

Where F is the fitness of string . This probabilistic interpretation is the reason why fitness should be positive.
If they arent guaranteed positive, then Boltzmann selection can be used to make them so (s is the selection strength)

Generating Offspring
After selection of the parents in the generation we proceed forward for generation of offspring We now decide how to combine their two string to generation of the offspring, which is genetics part of the algorithm. We have few genetic operators which are used to generate offspring

Genetic Operators
There are various genetic operators in practice Following are some of the operators that we will be discussing about 1. Crossover 2. Mutation 3. Elitism, Tournaments, and Niching

Crossover
It is analogous to reproduction and biological crossover, upon which genetic algorithms are based. Cross over is a process of taking more than one parent solutions and producing a child solution from them. we generate the new string as part of the first parent and part of second. There are methods for selection of the chromosomes. Those are also given below.
Single Point Crossover Multi Point Crossover Uniform Crossover

Single Point Crossover

A single crossover point on both parents strings is selected. All alphabets beyond that point in either string is swapped between the two parents. The resulting strings represents new children:

Multi Point Crossover

Multi Crossover point is selected. If there are two crossover point then it is two-point crossover Two-point crossover calls for two points to be selected on the parent strings.

Everything between the two points is swapped between the parent string, rendering two child strings

Uniform Crossover
The Uniform Crossover uses a fixed mixing ratio between two parents.

Unlike single and multi-point crossover, the Uniform Crossover enables the parent to contribute the gene level rather than the segment level.
If the mixing ratio is 0.5, the offspring has approximately half of the genes from first parent and the other half from second parent, although cross over points can be randomly chosen as seen below:

Crossover in String

Mutation
The exploitation of the current best strings is performed by the mutation operator which effectively performs local random search. Value of each element of string is changed with some (usually low) probability p. For knapsack problem, mutation causes a bit-flip, as is seen below

Mutation (contd..)
For chromosomes with real values, some random number is generally added or subtracted from the current value. Often P 1/L where L is the string length, so that there is approximately one mutation in each string. This might seem quite high, but it is often found to be a good choice given that the mutation rate has to trade off doing lots of local search with the risk of disrupting the good solutions.

Elitism, Tournaments, and Niching

Problem of standard GA: the best fitness can decrease as well as increase in the next generation. Reason: because the best strings in the current population are not copied into the next generation, and sometimes none of their offspring are as good. Solution: Elitism : simple idea of copying the best strings in the current population into the next population without any change. Tournament : two parents and their two offspring compete, with the two fittest out of the four being put into the new population

Elitism, Tournaments, and Niching (contd..)

Problem with Elitism and tournaments Elitism and tournaments both ensure that good solutions arent lost, they both have the problem that they can encourage premature convergence, where the algorithm settles down to a constant population that never changes even though it hasnt found an optimum. Exploration will be downplayed, making it hard to escape from the local maximummost of the strings will have worse fitness, and will therefore be replaced in the population. Eventually, majority of the strings in the population will be the same and represent the local maxima instead of global maxima

Elitism, Tournaments, and Niching (contd..)

Solution to the problem of Elitism and tournaments Niching : population is separated into the several subpopulation (islands of population), which all evolve independently for some period of time, so that they are likely to have converged to different local maxima and few members of one subpopulation are occasionally injected into another subpopulation It is also known as fitness sharing approach

This biases the fitness function towards uncommon strings, but can also mean that very common good solutions are selected against.

Basic Genetic Algorithm

Using Genetic Algorithm

Map Coloring It is typical discrete optimization problem. We want to color a graph using only k colors, and choose them in such a way that adjacent regions have different colors. It is mathematically proven that any two-dimensional planar graph can be colored with four colors, which was first ever proof that used a computer program to check the cases.

We will solve for 3 color problem.

Using Genetic Algorithm (contd..)

Steps for 3-color problem

1. Encode possible solutions as strings:

for this problem well choose our alphabet to consist of the three possible shades (black (b), dark (d), and light (l), say). So for a six region map, a possible string is = {bdblbb}

2. Choose a suitable fitness function:

the thing that we want to minimize is the number of times that two adjacent regions have the same color. We can turn it into the fitness function either by using the Boltzmann selection or to count the total number of lines between regions and subtract off the number where the two region on either side of the line have the same color.

Using Genetic Algorithm (contd..)

Using Genetic Algorithm (contd..)

In the figure above (fig 12.5) , it has 16 out of 26 boundaries correct, so its fitness is 16.

3. Choose suitable genetic operators

We will use standard genetic operators (crossover and mutation) for this

Having made those choices, we can let the GA run on the problem, with a possible population and their offspring shown in figure in next slide, and look at the best solutions after some present number of iterations.

Using Genetic Algorithm (contd..)

Using Genetic Algorithm (contd..)

Knapsack Problem

Output of the knapsack problem that we defined earlier in the slides. We use same fitness function that we defined earlier.

Using Genetic Algorithm (contd..)

Punctuated Equilibrium For a long time, one thing that creationist and other who did not believe in evolution used as an argument against it was the problem of the lack of intermediate animals in the fossil record. Argument seems to be correct because there is no record of any species between the transition of human from man. But GA demonstrate solution to this problem or argument and explains why argument is incorrect. The evolution actually seems to work is known as punctuated equilibrium.

Using Genetic Algorithm (contd..)

Punctuated equilibrium states that there is basically a steady population of some species for a long time, and then something changes and over a very short (in evolutionary terms still hundreds or thousands of years) period, there is a big change, and then everything settles down again. So the chance of finding fossils from the intermediary stage is quite small. The graph in next slide shows this effect.

Using Genetic Algorithm (contd..)

Limitations of GA
GA can be very slow. It may take longer to escape from the local maxima.

There is no absolute assurance that a genetic algorithm will find a global optimum
because we dont know anything about the fitness landscape, we cant see how well the GA is doing basic criticism of genetic algorithms is that it is very hard to analyze the behavior of GA

Neural Network & GA

GA can be used to train the neural network. We can encode the problem of finding the correct weights as a set of strings, with the fitness function measuring the sum-of-squares error.

GA can be used to choose the topology of network. Mutation is used in finding the most suitable topology which includes following mutations
Delete a neuron Delete a weight connection Add a neuron Add a connection

Genetic Programming
introduced by Jhon Koza combines the idea of machine learning and evolved tree structures. tree-based variants on mutation and crossover are defined like replace sub-trees by other sub-trees, either randomly generated or swapped from another tree, and then the genetic program runs just like a normal genetic algorithm, but acting on these program trees rather than strings

Genetic Programming (contd..)

Mutation in Genetic programming + 0 +

Genetic Programming (contd..)

-

Offspring after mutation

1 x
1 1 *

Genetic Programming (contd..)

Example of crossover + 0

0 x

1 +

1 +

*
1 x x x *

Genetic Programming (contd..)

+

3
1 *

x x x

2x-3
Athematic trees x

X2+1

Genetic Programming (contd..)

Genetic programming has been used for many different tasks, from recognizing skin melanomas to circuit design, and lots of very impressive results have been claimed for it.

The search space is unbelievably large, and the mutation operator not especially useful, and so a lot depends upon the initial population.
Set of possibly useful sub-trees are usually chosen by the system developer first in order to give the system a head start.

Combining Sampling with evolutionary learning

Most basic version is known as PopulationBased Incremental Learning (PIBL), and it is amazingly simple. It works on a binary alphabet, just like the basic GA, but instead of maintaining a population, instead it keeps a probability vector p that gives the probability of each element being a 0 or 1.

Initially, each value of this vector is 0.5, so that each element has equal chance of being 0 or 1.

Combining Sampling with evolutionary learning (contd..)

A population is constructed by sampling from the distribution specified vector, and the fitness of each member of the population is computed. A subset of this population (typically just the two fittest vectors) is chosen to update the probability vector, using a leaning rate , which is often set to 0.005

where best and second represent the best and second-best elements of the population

Combining Sampling with evolutionary learning (contd..)

The population is then thrown away, and a new one is sampled from the updated probability vector.