You are on page 1of 5

Parallel Genetic Algorithm Based on a New Migration Strategy

Leila Falahiazar
Department of computer Engineering Science and Research Branch, Islamic Azad University Tehran, Iran leilalfa@gmail.com

Mohammad Teshnehlab
Department of computer Engineering Science and Research Branch, Islamic Azad University Tehran, Iran Teshnehlab@eetd.kntu.ac.ir

Alireza Falahiazar
Department of computer Engineering Science and Research Branch, Islamic Azad University Tehran, Iran arfa9000@gmail.com

Abstract—Parallel Genetic Algorithm (PGA) is used in many practical global optimizations to achieve high speed in convergence. The Island Model Parallel Genetic Algorithm (IMPGA) is very useful. Genetic Algorithms are one of the most powerful search and optimization method when we must solvecomplex and time consuming problems. IMPGA are more flexible than other PGA methods. There are several variables in the IMPGA that determining them are effective to enhance performance of theIMPGA. In this paper, we proposed a Migration method (Max-Min method). In our proposed method, according to status of subpopulation and comparing subpopulation with other subpopulations, the individuals for migrationare selected. In addition to enhancing the performance of PGA, we propose another method that embedding HillClimbing Algorithm within the structure of the PGA. As we know, creating an optimized structure for a Neural Network is a time consuming problem and costly one. The problem was studied in this paper is to determine the structure of a Neural Network forforecasting next day air quality. In addition, we used real data which was received from the Meteorological Organization and Tehran's Air Pollution Company. Output of the neural network is the value of Ozone Gas (o3) for the next 24 hours. The results of our two proposed methods are compared with conventional methods in other papers. Our algorithm has better performance than other papers. Keywords- Genetic Algorithm; Parallel Genetic Algorithm; Island Model Parallel Genetic Algorithm; Neural Network;

the entire population divides into some subpopulation or Island, Each island works the simple GA alone, after several generations, individuals should be selected from the source island and according to the criteria, the selected individuals should be send to other island . If we are concerned about the migration interval and the number of people who are selected for migration[1], the migration interval should be performed with low frequency and the number of individuals who are selected for migration, is a small number of subpopulation. Therefore, an optimal method in order to determine how migration rate and How to migrate is an important task. In this paper, section ȱȱ reviews the research on the migration parameters, Section ȱȱȱ Proposed the Max-Min method, The second method proposed in Section ȱV that is how HillClimbing algorithm is embedded in PGA. In Section V, simulations of the optimization of the neural network predictor are presented to confirm the efficiency of the Max-Min and Hill-Climbing approach and the results are analyzed. Section Vȱ concludes the paper. II.BACKGROUND The Island Model Parallel Genetic Algorithm also called coarse-grained. The PGA is very complex because it has many variables. The variables in IMPGA is consist of: migration rate, migration interval, Selection individuals formigration and other simple variables in genetic algorithms such as mutation rate, crossover rate and parent selection[2].IMPGA optimization algorithms is one of the best optimization method, on the other hand, this optimizationmethod is used in many papers. Impact of migration in IMPGA is very important, because it has great influence on the convergence or the premature convergence[1]. As well as, numerous articles about mutation rate, crossover rate and parent selection is provided for IMPGA, An adaptive method for determining the mutation rate and crossover rate according to the individuals that they are in high or low fitness area, was provided in article[3], furthermore to increase exploration and exploitation, this article present changing on the mutation rate, the crossover rate and parent selection. Diversity is more important for the performance of PGA. In

I.INTRODUCTION Parallel Genetic Algorithm (PGA) is used in many practical issues and the Island Model Parallel Genetic Algorithm (IMPGA) is very useful, for this reason, Many papers have investigated the problems of IMPGA and optimized IMPGA. Determine the variables of IMPGA is the problem of this method , the variables in IMPGA is consist of: migration rate, migration interval, Selection individuals for migration from source island and Other simple variables in genetic algorithms are mutation rate, crossover rate and parent selection. In this article, we discuss about migration rate and we investigate the research on this issue. Migration refers to movement of individuals every few generations between the islands. The island model is made of several islands, Initially,

978-1-4673-0255-5/12/$31.00 c 2012 IEEE

37

If maxi is smaller than the maxj(j=1. More precisely.2. Figure 1. the impact reasons are. because the selection of individuals from the pool migration based on the following two rules: 1) Maximum fitness migrates: Elitism migration used for the i-th island. In[5] .[4]. the diversity mechanism was fully studied. Max-Min algorithm is used as a migration step is shown in IMPGA. the i-th island accepts an elite from the other islands that this elite fitness is greater than the maxi. each island receive a maximum and a minimum. the super-linear speed-up caused by the information exchange and the second one. and in this article studied on what topology must use between the islands. 2) After collecting all the individuals of the islands to determine which individuals should Migrate to which island . a fuzzy method for determining the migration rate.n). this migration method will have a greater impact on convergence.…. The communication (relation) between the islands has a great importance. In MaxMin method when the migration interval occurs.2. In general. select an appropriate topology is important for problem solving. In the final stage of the migration. In the[6]and[7]were investigated who must migrate from the source island to other island. the Max-Min Method determines which individuals will be selected from the pool migration and migrates to the destination island. after that... Migration topology affects the final solution quality and speed of convergence.. according to the Max-Min rules mentioned above must act: a. in our method. the two rules was described above to perform in this manner: (The i-th island (i =1. Maxi –Avragei<<low(1) 38 2012 International Conference on Recent Advances in Computing and Software Systems .some of the major topologies was mentioned and it was explained the impact migration on the topology optimization. b. we optimize the rate of migration. selection of individuals is elitism or random and migration the elitism to other islands was suggested. the first one. In this article we have proposed a Max-Min method. In addition. all of the maximums and minimums are placed within the pool migration. Maximum and Minimum fitness of each island select for migration. which individual should be selected from the source island and migrate to the destination island. we provide a better way to do a migration between the islands. the migration rate and selection of individuals for migration based on the status of island and compared the status of island with other islands. mutation rate and crossover rate was presented. the convergence of the island (or subpopulation) is based on the global maximum fitness and the average individual fitness. In this time.i+2. (The generation is repeated several times and then Islands must communicate each other). then destination island replaces randomly the maximum and the minimum with its individuals and finally migration ends. communication may be either unidirectional or bidirectional. IMPGA method is widely used and improving this method is more important.. Our proposed method is called Max-Min. the minimum fitness from other islands must inject into the i-th island until diversity could increase. The migration and GA parameters were studied in articles [8]and[9]. In our proposed method. consequently. Thus. III. when the migration.PROPOSED THE MAX-MIN MIGRATION METHOD Determining the variables in the IMPGA always is a problem.i-1.avrageiandmini(the maximum fitness of the i-th island =maxi=elitei)( the average fitness of the i-th island =avragei)( the minimum fitness of the i-th island =mini=worsti). when the migration interval occurs. n) wants individuals to accept them as immigrants. The paper focuses on the convergence of each islands to determine the variables of IMPGA and the migration rate obtained of status of each island. 2) Minimum fitness migrates: If the distance between maximum fitness and average fitness (in the i-th island) is low. we must determine. if distance between maxi and avragei is very small. hence. In this paper. we propose a Hill-Climbing algorithm to enhance performance of the PGA algorithm. we create a pool migration. determining the migration rate is only considered the convergence of each island that the obtaining convergence of each island is separate from the convergence of other islands . diversity was reduced and diversity must increase. so that) 1) Each island send three fitness to the migration pool: maxi.….

Figure 2. Data set received from the Weather Organization and Air Pollution Company in Tehran during the years 2006–2009. In this paper. We propose that if the best fitness did not change in several generations. the crossover operator is removed from the GA and the mutation operator until it was acted on individual genes. Significantly raise the performance of the PGA algorithm. now the mutation operator acts on only a few genes randomly.1 or smaller. the PGA algorithm performance upgrade and our method will help to become PGA more efficient. the diversity of the island has reached its lowest possible. parents selection is random). Low value of the formula (1) is related to the problem which implements in the PGA. Thus. Inputs of the forecasting air pollution system are all features of Weather such as wind speed. Thus. In our proposed method. 10% of the total number of generations). they try to achieve a higher convergence. we embed the Hill-Climbing algorithm into the PGA that to enhance performance of the PGA algorithm. In this paper. the simulation is a neural network. . As shown in Fig. they use the Hill-Climbing algorithm[10]. Feature of neural network to be selected in this study: Conjugate Gradient Back-Propagation training algorithm and Multi-Layer Perceptron Neural Network and Levenberg-Marquardt back propagation training function for this network is intended. In fact. the simulation in section V confirm this result. When Hill-Climbing algorithm is implemented in PGA. O3. because this system is too chaos and non-linear phenomenon. wind direction. with the Hill-Climbing algorithm in every generation. the crossover is removed and the mutation is changed from bit to bit. minj is selected from the pool migration which is smaller than the mini and accidentally.g. the HillClimbing algorithm looks for a more efficient chromosome instead of looking for the best chromosome after the PGA finished. but determining the network structure is still difficult. and it have high dimensional sample space. in most articles. flowchart of embedding The Hill-Climbing Algorithm into the PGA V. We used Article[12] for eliminating the problem of missing values in air quality data sets. it will necessary change the mutation and crossover operators. neural network training would be very time consuming. to a few bits into the chromosome. we set the constant value for variables in the PGA (the mutation rate and the crossover rate are constant. Fig. we have investigated an application.1. the fitness value is the neural network's error. minj is replaced with an individual of the i-th island. In fact. the migration is finished and PGA continues again. the process of modeling real world as air quality is a difficult task[11].. According to Fig. then the i-th island must accept a minimum fitness of the other islands. crossover operator enhances the exploitation and mutation operator helps to exploration[3]. After that.2 illustrates how the Hill-Climbing algorithm is embedded in the PGA. When the distance between the max fitness and average fitness is equal to 0. 2012 International Conference on Recent Advances in Computing and Software Systems 39 . Consequently. the migration operation would change. with minor changes in the final chromosome. we use GA as a feature selection.The diversity of the i-th island has decreased.. Generally. if after several generations.. if the best fitness in the population does not change in every some generations (e.. maximum fitness dose not changed. EMBEDDING HILL-CLIMBINGPGA The goal in all the research on the IMPGA algorithm is enhanced performance. in our method.. Although neural networks have been used successfully in this domain. As we know.2. and it is time consuming.SIMULATION To evaluate the proposed method.. to determine the structure of the neural network and optimization of neural network inputs. because of this function is faster than all the functions in the MLP and training each of neural network structuresis very time consuming (1000 epochs). in each generation.et. with small changes in genes of chromosome can reach a final solution.1. and air pollutions such as NO2. we will actually achieve a solution that is very close to the final answer. The algorithm flowchart of Max-Min proposed in this paper is shown in Fig. So. Each chromosome consists of a neural network structure and neural network's input data. In fact. So. IV. after the PGA algorithm ended. In addition.

Other variables of PGA are as follow: The total number of individuals is 45 and every island consists of 15 individuals. C. . Chromosome structure is shown in figure 3: Figure 3. Chromosome representation This chromosome is a binary type and the encodings are gray. Determining the fitness of each chromosome in the simulation Each chromosome represents the neural network structure. worst of their fitness to the migration pool and based on two rules of Max-Min (previouslydescribed). Type of the crossover is the double point. Destination island accepts individuals and randomly exchanges with one of the existing individuals. determining the fitness of each chromosome can be obtained from the following formula[11]. and the mutation has to flip all n bits (n = 37).A. Migration interval is done every five generations [1]. This Chromosome makes the neural network structure and selecting the neural network inputs.

which can be from 1 to 31. Topology Topology used in this simulation is typed of fully connection and islands communication is bidirectional (Fig. Comparing the result of the Max-Min &Hill-Climbing method with FASPGA method and a pure PGA. 0.35 0.45 0. Simulation results Simulation results were presented by comparing our proposed Max-Min method with the FASPGA method (provided in articles[8] and[9]).3 chromosome has four sections (from left to right): 1. the fitness function achieved by the formula (3) (N = 5). As described in the proposed method.25 0. Topology used in the simulation. Figure 5. b1: the first bit is the number of hidden layer in neural network.5.7 illustrate results of simulations. 2. ƿ parameter represents the average amount of data has been viewed 5 times. 4. The value of fitness function (F) tends to zero and final goal is minimize the value of network error. These five bits indicate the number ofsecond hidden layer of the neural network. 3.3 0.2 0 10 20 30 40 50 60 generation 100 70 80 90 100 Figure 4. b2 to b6: Here we have five-bit which is gray cod. ൌ ͳ െ ൤σొ ഥȁାȁ୭౟ ି୭ ഥȁሻమ ౟సభሺȁ୮౟ ି୭ σొ ౟సభሺ୮౟ ି୭౟ ሻ According to fig.4 " Best fitness " 0. B12 to b37: The number of inputs that can be from 1 to 26. else if b1 equal to zero then existing one hidden layers. value of zero means the absence of the neural network inputs and value of one means the presence of the neural network inputs. In fact. if b1 equal to one then existing two hidden layers. These five bits indicate the number of first hidden layer of the neural network. D. Finally. all the islands sent the best and 40 2012 International Conference on Recent Advances in Computing and Software Systems . Fig. ൨ (2) (3) ൌ σ୒ ୧ୀଵ ୒ The fitness of the neural network is acquired by predicted and observed values based on the Index of Agreement formula (IA) and N parameter in this formula is the number of neural network's test.6 and Fig. the migration is done in some generations. B. b7 to b11 : Here we have five-bit which is gray cod. which can be from 1 to 31. and the Pure Parallel Genetic Algorithm. Fig.4).5 Max-Min&Hill-Climbing PGA FASPGA pure PGA ሺଵି୍୅ሻ 0. test is done 5 times for each chromosome and Oi parameter represents the actual value of Ozone gas and Pi parameter represents predicted value of Ozone gas by the neural network.

35 0. "Fuzzy adaptive search method for parallel genetic algorithm with island combination process. Introduction to Genetic Algorithms. solving the time consuming problems easily. 2003. Figure 6. Biscani. Ari Karppinenb. 2007.5 Max-Min PGA FASPGA pure PGA 0. Genetic Algorithm is one of the very applicable and powerful tools in search techniques and optimization. Mikko Kolehmainen Harri Niskaa. "On the Impact of Migration Parameters on DIMEP for Designing Combinational Circuits.0. Oliveto." Elsevier.*. [8] Qiang Li and Yoichiro Maeda. Comparing the result of Hill-Climbing PGA method with the FASPGA method and a pure PGA. 0. and Dirk Sudhol. 2005. "On the impact of the migration topology on the Island Model. and Qiang Li." IEEE. 2004.2 0 10 20 30 40 50 60 generation 100 70 80 90 100 [2] S. Juhani Ruuskanena. IMPGA works very powerful for optimizing problems. "Methods for imputation of missing values in air. and L. Masahide Ishita. Srinivasa.N. 2007. "A self-adaptive migration model genetic algorithm for data mining applications. [11] *." 2008 IEEE.. "Theoretical Analysis of Diversity Mechanisms for Global Exploration. Pietro S.G.45 [5] M.Deepa.25 0. Venugopal." 2008. 2008.The simulation results. Verlag Berlin Heidelberg: Springer. we have decided that the Max-Min algorithm can be used for optimization the fuzzy Systems and providenew solutions to eliminate problems of the fuzzy systems.45 0. Dortmund: IEEE. "Evolving the neural network model for forecasting air pollution time series.2 0 10 20 30 40 50 60 70 80 90 100 Figure 7." Elsevier. 0. consequently. Artificial Intelligence: A Modern Approach. and Mustapha Kamel Rahmouni. 2007. VI. REFERENCES [1] Zbigniew Skolicki and Kenneth De Jong." Elsevier. 2004. Comparing the result of Max-Min method with the FASPGA method and a pure PGA.R. clearly shows that the Max-Min method and combining the Max-Min and the Hill-Climbing have better performance on obtaining the best solution than the FASPGA and pure PGA. [6] Fatima Zohra Hadjam. K.3 0." Elsevier. Juhani Ruuskanena. D.N.5 Hill-Climbing PGA FASPGA pure PGA 0. Kari Tuppurainenc.4 0. Cause.25 [9] Yoichiro Maeda. Teri Hiltunena. [7] Plamenka Borovska and Milena Lazarova. Heikki Junninena. 0. Izzo. Patnaik.35 0. Rucin'ski. 0. [3] K. As you consider. Russell and Peter Norvig. optimizing and eliminating problems of the IMPGA. the Max-Min method tend to optimize solutions slowly and shows better results than the FASPGA method and the pure PGA. [10] Stuart J.M. [12] b. "Distributed Adaptive Search Method for Genetic Algorithm. 2005. For future work." Elsevier.4 " Best fitness " 0. [4] Tobias Friedrich. When we are faced 2012 International Conference on Recent Advances in Computing and Software Systems 41 .3 with the problems of time consuming. "The influence of migration sizes and intervals on island models. Harri Niskaa. and F. CONCLUSION In this article we proposed the Min-Max method that is a new method of migration for Island Model Genetic Algorithm." ACM. Claudio Moraga. Migration Policies for Island Genetic Models on Multicomputer Platform. 2010.Sivanandam S.