Ant Colony Optimization: A New Meta-Heuristic Mareo Dorigo TRIDIA, Université Libre de Bruxelles mdorigo@ulbac.be the solution of dificult discrete optimization problems. In this paper we put these algorithms in a common frame- work by defining the Ant Colony Optimization (ACO) ligmatic examples of ap- ns of these novel meta-heuristic are given, as well asa brief overview of existing applications. 1 Introduction In the early nineties an algorithm called ant system was proposed as a novel heuristic approach for the solution of combinatorial optimization problems (Dorigo et al, 1991; Dorigo, 1992; Dorigo etal, 1996). Ant system (AS), which ‘was first applied to the traveling salesman problem, was re cently extended and/or modified both to improve its perfor- ‘mance and to apply it to other optimization problems. Im- proved versions of AS include, among others, ACS (Dorigo| ‘& Gambardella, 1997), MAX-MZIN Ant System (Stitzle & Hoos, 1998p), and ASran4 (Bullnheimer et a, 19976), {All hese algorithms have been applied to the TSP with vary: ing degree of suocess, but always improving over AS perfor ‘mance. These improved versions, as well as the original AS, have also been applied toa diverse set of optimization prob- lems. Examples are quadratic assignment (Maniezzo et al, 1994; Stitzle & Hoos, 1998), vehicle routing (Bullnheimer etal, 19974; Gambardella etal, 1999), connection oriented, and connectionless network routing (Schoonderwoerd er al., 1996; Di Caro & Dorigo, 1998; Di Caro & Dorigo, 1998). sequential ordering (Gambardella & Dorigo, 1997), graph coloring (Costa & Hertz, 1997), shortest common superse- quence (Michel & Middendorf, 1998), single machine tad ress (Bauer etal, 1999), multiple knapsack (Leguizamén & Michalewiee, 1999), ete, For many of these problems, there sults obtained place the ant system based approaches among, the best available heuristics. In this paper we present the Ant Colony Optimization (ACO) meta-heurstc, that is, the result of an effort 10 de- Fine a common framework forall these versions of AS. The ‘main motivations for defining the ACO meta-heurstic are the desice to provide a unitary view of the ongoing research in this growing field, as well as the hope that such a charac- terization will help to identify the most important aspects of these algorithms and therefore make the development of new applications easier 0-7803-5536-9/99/510.00 ©1999 IEEE Gianni Di Caro IRIDIA Université Libre de Bruxelles dicaro@iridia.ub.ac-be 2 The ACO meta-heuristic The ACO meta-heuristc ean be eppied to discrete optimiza tion problems characterized as follows © C= (e102)... sen} is finite st of components Aleve | (Cics) € Cy [LI SN isa finite set of. possible connectionsfransitions among the elements of , where Cis subst ofthe Cartesian product C x C. Je, © I{leejt) 8a connection cost function associ- ated wo each Lc, € L, possibly parameterized by some time measure i = MC, Lt) isa finite set of constraints assigned ‘over the elements of © and D. 8 = (Case), .-,6ks---) isa sequence over the elements of C (or, equivalently, of L). A sequence s is also called a state ofthe problem. If Sis the set of all pos- sible sequences, the set 5 of all the (subsequences that are feasible with respect tothe constraints (0, L,t), isasubsetof S. The elements in $ define the problem's feasible states. The length ofa sequence s, that i, the ‘number of components in the sequence is expressed by Ish Given two states 8; ands a neighborhood structure is defined as follows: the states is said to be a neighbor of s1 if both sx and so are in S, and the state s2 can be reached from sy in one logical step (that is, if e1 isthe last component in the sequence determining the State sy, it must exists ca € C such that ye, € ands» = (s1,¢2))- ‘The neighborhood of a sates is denoted by N, {isa solution if itis an element of S and stisties all the problem’s requirements. A muli-dimensional solu- tion is a solution defined in terms of multiple distinct sequences over the elements of C. Jo(Lyt) is @ cost associated to each solution y. ‘Jo(Evt) i a function of all the costs Ja, ofall the connections belonging tothe solution ¥. ‘Consider the graph = (C, L) associated toa given discrete ‘optimization problem instance as defined above, The solu tions to the optimization problem can be expressed in terms ‘of feasible paths on the graph G. ACO algorithms can be used to find minimum cost paths (sequences) feasible with respect to the constraints 2. For example, inthe traveling salesman problem, Cis the set of cities, L isthe set of ares connecting 1470cities, and a solution is an Hamiltonian circuit. ‘ACO algorithms, that is, instances of the ACO meta- heuristic introduced in the following of this section, use a ‘population of ants to collectively solve the optimization prob- lem under consideration by using the above defined graph representation. Information collected by the ants during the search process i stored in pheromone tails 7, associated t0 connections ls. (Here and in te following, we simplify no- tation by refering fo fa hy.) Pheromone trails encode a long-term memory about the whole ant search process. De- pending on the problem representation chosen, pheromone trails can be associated tall problem's arcs, or only to some ‘of them. Arcs can also have an associated heuristic value my representing a prior information about the problem instance definition or runtime information provided by a source dif ferent from the ans ‘Ants of the colony have the following properties: ‘An ant searches for minimum cost feasible solutions fy = ming Jo(E,). ‘© An ant & has a memory M' that it ean use to store information on the path it followed so far. Memory can be used to build feasible solutions, to evaluate the solution found, and to retrace the path backward ‘© Ananth in state s- = (s--1, i) can move to any node jin its feasible neighborhood Nf defined as N¥ = IGEN) A (s3) € 5)} ‘An ant & can be assigned a start stare sf and one or more termination conditions e. Usualy, the start state is expressed as a unit length sequence, thats, a single component. ‘Ants start from the start state and move to feasible neighbor states, building the solution in an incremen- tal way. The construction procedure stops when for at leastone ant kat least one of the termination conditions tis satisfied ‘An ant k located on node i can move toa node j chosen in NS. The move is selected applying a probabilistic decision rule ‘© The ants’ probabilistic decision rule is a function of (i) the values stored in a node local data structure Ay [ay called ant-rowing table, obtained by a functional composition of node locally available pheromone trails and heuristic values, (i) the ants private memory stor- ing its past history. and (ui) the problem constraints. ‘© When moving from node é to neighbor node j the ant ‘can update the pheromone trail 743 on the are (i,j) ‘This is called online step-by-step pheromone update ‘© Once built a solution, the ant can retrace the same path backward and update the pheromone trails on the tra- versed arcs, This is called online delayed pheromone update ‘© Once it has built a solution, and, ifthe case, after it has retraced the path back to the source node, the ant dies, freeing all the allocated resources. Although each ant of the colony is complex enough to find «feasible solution to the problem under consideration, good {quality solutions can only emerge as the result of the collee- tive interaction among the ants. Each ant makes use only of private information and of information local to the node itis visiting (note that we use the terms node and compo- nent, as well as arc and connection/transition, as synonyms), and communication among ants is indirect and is mediated by the information they read/vrite in the variables storing pheromone trail values. Ants adaptively modify the way the problem is represented and perceived by other ans, but they are not adaptive themselves. Informally, the behavior of ants in an ACO algorithm can be summarized as follows. A colony of ants concurrently and asynchronously move through adjacent states of the problem by moving through neighbor nodes of G. They move by ap- plying a stochastic local decision policy that makes use of the information contained in the node-local ant-routing ta- bles. By moving, ants incrementally build solutions to the ‘optimization problem. Once an ant has built a solution, or While the solution is being built, the ant evaluates the (partial) solution and deposits information about its goodness on the Pheromone trails ofthe connections it used. This pheromone information will direct the search ofthe future ans Besides ants’ activity, an ACO algorithm includes two more procedures: pheromone trail evaporation and daemon ‘actions. Pheromone evaporation is the process by means of Which the pheromone trail intensity on the connections au- tomatically decreases over time, From a practical point of View pheromone evaporation is needed to avoid a too rapid convergence of the algorithm towards a sub-optimal region implements a useful form of forgeting, favoring the explo- ‘ation of new areas of the search space. Daemon actions, that are an optional component of the ‘ACO meta-heuristic, can be used to implement centralized actions which cannot be performed by single ants. Examples are the activation of a local optimization procedure, or the collection of global information that can be used to decide whether itis useful or not to deposit additional pheromone to bias the search process from a non-local perspective. As a practical example, the daemon can observe the path found by ‘each ant in the colony and choose to deposit extra pheromone fon the ates used by the ant that made the shortest path Pheromone updates performed by the daemon are called off line pheromone updates. InFigure 1 the ACO meta-heuristc is described in pseudo- code, ‘The main procedure of the ACO meta-heuristic manages, via the schedule-activities construct, the scheduling of the three above discussed components of an ‘ACO algorithm: ants’ generation and activity, pheromone ‘evaporation, and daemon actions. It is important to note that the schedule activities construct does not spec- ify how these thee activities are scheduled and synchronized and, in particular, whether they should be executed in a com- 1471+ proadare ACO.mtshertel) The (emis rte 1 “Tipnestomentaci [ewe tune ‘Dims oe) + procadare et geon and con) We ortstaste resource) came neon) fre enact) fe ters) ant Ite tno Beep nema ieleraeepet seers) 1c ite ahemontn sta) eS trun sale igure 1: The ACO mtateisie nthe ACO-metacheusistic() rove, be procedire daenon.act ons) is 6) opin. When (SShcmcnad few comand sence by saz pose tag wotl knowledge lhe newactive-ant() procedure, he a= Greccstav (ae 4) refer toa complet soon Dalby thea, We yp ad eps pce sng pte at 9.10 Deromone depot byte damon. Comments re loan rae, pletely parallel and independent way, or if some kind of syn- ctronization among them is necessary. This leaves the de- signer the freedom to specify the way these three procedures should interact. Although ACO algorithms are suitable to find minimum cost (shortest) paths on a graph in genera, it is important to note that they ae an interesting approach only for those short- est path problems to which more clasical algorithms like dy- namic programming or label correcting methods (Bertsekas, 1995) cannot be efficiently applied. This is the case, for ex- ample, for the following types of shortest path problems: '» NP-hard problems, for which the dimension of the full state-space graph is exponential in the dimension ofthe problem representation, In this case, ants make use of the much smaller graph G, built from the problem’s ‘components, and use their memory to generate feasible solutions which in most ACO implementations are then taken to a local optimum by a problem specific local optimizer. ‘© Those shortest path problems in which the properties of the problem’s graph representation change overtime concustetly with the optimization proces, tat has 12 adapt tothe problems dynamics. In this ease, the rob lems graph can even be physically avalable (like in networks problems), but its propertis, ike the value of connection ests J, t),canchange over time. In this ase we conjecture thatthe use of ACO algorithms be- comes more and mote propriate as the variation rate of costs Jac, (t) increases andlor the knowledge about the variation process diminishes. “Those problems in which the computational architec ture is spatially distributed, asin the case of parallel andlor network processing. Here ACO algorithms, due to ther intinsically distributed and mult-agen nature that well matches these types of architectures, ean be very effective. In the following section we will consider the aplication of the ACO-metacheuristic to two paradigmatic problems be- longing o the above defined classes of problems the ta eling salesman problem (TSP) and adaptive routing in com ‘munications networks, TSP isthe prototypical representative of NP-hard combinatorial optimization problems (Garey & ohnson, 1979) where the problem instance is statically as- signed and the information is globally available. On the con- trary, inthe problem of adaptive routing in communications networks an exogenous process (the incoming data traffic) rakes the problem instance change overtime, and temporal constraints impose o solve the problem in a distributed way. 3 ACO for the traveling salesman problem Using the terminology introduced in the previous section, the traveling salesman problem can be defined as follows. Let be a set of components, representing cities, L be a set of connections fully connecting the elements in C, and Jo, be the cost (length) ofthe connection between cy and cy, that i, the distance between cities {and j. The TSP is the problem of finding a minimal length Hamiltonian cieuit on the graph G = (C\L). An Hamiltonian citcuit of graph G isa closed tour ¢ visiting once and only once all the Ne: nodes of G. Its length is given by the sum of the lengths Jee ofall the ares ‘of which itis composed. Distances may be asymmetric: We hhave then an asymmetric TSP in which Jey, can be different from Je, Also, the graph need not be fully connected: if it is not, the missing ares can be added giving them a very high length. “The traveling salesman problem plays a central role in ant colony optimization because it was the fist problem to be at tacked by these methods (See (Dorigo, 1992; Dorigo et al, 1991; Dorigo et a., 1996)). Among the reasons the TSP was chosen we find the following ones: (j) itis relatively easy to adapt the ant colony metaphor to i, (i) itis @ very dif ficult problem (NP-hard), (i) itis one of the most studied problems in combinatorial optimization (Lawler etal, 1985; unReinelt, 194), and (i) itis very easy to state and explain it $0 thatthe algorithm behavior is not obscured by too many technicalities. In the following we will present Ant System, a paradig- ‘atic example of how ACO algorithms canbe applied tothe TSP. Extensions of Ant System can be found, for example, in @ullaheimer er al, 1997; Dorigo & Gambardells, 1997; Stitzle & Hoos, 19980). ‘Ant System (AS) can be informally described as follows ‘A number m of ants is positioned in parallel on m cities. The fants” start state, that is, the stat city, canbe chosen randomly, and the memory M' ofeach ant is initialized by adding the current start city tothe set of already visited cites (ntally empty). Ans then enter eycle (Figure I ines 4 + 12 ofthe new.active.ant () procedute) which lasts No iterations, thats, until each ant has completed a tour During each step an ant located on node i considers the feasible neighborhood, reads the enties a's of the antrouting table A, of node + (Figure 1, line § of the new.active.ant () procedure), computes the transition probabilities (ine 6), and then applies its decision rule to choose the city 10 move to (ine 7), moves tothe new city (ine 8), and updates its memory line 1). Note that, because in AS there is no online step-by-step pheromone update in- structions at lines 9 and 10 of the newactive.ant () pro cedure of Figure | are not executed. ‘Once ants have competed a tour (which happens sy hronously, given that during each iteration ofthe while loop ‘ach ant adds new city tothe tour under construction), they Use their memory to evaluate the built solution and to e- trace the same tour backward and increase the intensity of the pheromone trails of visited connections ly (Figure 1, lines 13 — 16) This ha the effet of making the visited con. nections become more desirable fr future ants. Then the ants dio, fesing all the allocated resouees. In AS all the ants de- posit pheromone and no problem-specific daemon actions are performed. The triggering of pheromone evaporation hap- pens ater all ants have completed their tous. Of course, it ould be easy to add a leal optimization daemon action ike 4 3.opt procedure (Lin, 1965); this has been done in most of the ACO algorithms forthe TSP that have been developed as extensions of AS (ee for example (Dorigo & Gambardella, 1997; Stee & Hoos, 1998). The amount of pheromone tal 7, (#) maintained on con- nection fy is itended to represent the learned destailty of choosing ity j when in ety ¢ (which also corresponds tothe desirability that are ly belongs tothe tour built by an an} ‘The pheromone tail information is changed during prob- Jom solution to reflect the experience acquired by ants during problem solving. Ants deposit an amount of pheromone pro- Portiona tothe quality of the solutions y they produced: the Shorter the tour generated by an ant, the greater the amount of pheromone it deposits on the aes which i sed to gener- ate the tour. This choice helps to direct search towards good solutions, The memory (or internal state) M* of each ant contains the aleady visited cities. The memory .M* is used to define, for each ant k the set of cites that an ant located on it stil has to visit. By exploiting (* an ant kcan build feasible so- lutions, that is, ican avoid to visita city twice. Also, memory allows the ant o compute the lenath of the tour generated and to cover the same path backward to deposit pheromone onthe visited arc. The ant-routing table A, = [a(t] of node i, where Ni is the set of al the neighbor nodes of node i, is obiined by the following functional composition of pheromone tril 7.) and local heuristic values, Irg(0I in? Veale nal® where and are two parameters that contol the lative weight of pheromone tral and heuristic value. The heuristic values used are ny = 1/Jey» Where Ja, isthe distance between cities and . Inother words the shorter he distance betiveen two cities + and 3, the higher the heuristic value n ‘The probability pf,(t) with which atthe rth algorithm it eration an ant located in city chooses the city j € Ni to move ts given by the following probabilistic dessin rule: a of) ¥ ante) where Nf © Ni is the feasible neighborhood of node i for an k (hats, the set of cites ant k has ot yet visited) as ‘defined by using the ant private memory M¥ and the problem constants ‘The role ofthe parameters @ and f isthe following. If a = 0, the closest cites are more likely tobe selected: this corresponds 1 a classical stochastic greedy algorites (with multiple starting points since ans are initially randomly dis- tributed on the nodes). If on the contrary 8 = 0, only pheromone amplification is at work: this method will lead to the rapid emergence of a stagnation, that is, a situation in which all ants make the same tour which, in general is, strongly sub-optimal (Dorigo etal, 1996). An appropriate trade-off has tobe st between heuristic value and tail inte sity ‘Afterall ants have completed their tour each ant & de posits a quantity of pheromone Ar*(t) = 1/J§(¢) on each connection 1, that it has used, where J§(¢) isthe length of tour y*(t) done by ant k at iteration t: r= @ rylt) ryt) ArKO), Why EVA), °) where the number m of ants at each iteration is maintained constant) and is set to m = No. These parameters setings, as well asthe settings @ = 1, 6 = 5 and p = 0.5, were experimentally found to be good by Dorigo (Dorigo, 1992), 1473‘This way of setting the value Ar*(t) makes ita function of ‘he ant’s performance: the shorter the tour done, the greater the amount of pheromone deposited. ‘ter pheromone updating has been performed by the ans, pheromone evaporation is tiggered the fllowing rule is applet all the ars ofthe graph ral =~ Aral) ® ‘where p € (0,1 isthe pheromone tril decay cocicient (the inital amount of pheromone 7, (0) is set toa small positive constant value 79 on all re), 4 ACO for routing in communications networks ‘The generic routing problem in communications networks can be informally stated as the problem of building and us- ing routing tables to direct data traffic so that some measure of network performance, function of the network type and of the services provided, is maximized, ‘We can use the terminology introduced in Section 2 to give formal definition of the routing problem. Let the sets C and _L correspond respectively tothe sets of processing nodes and fof communication links of the real network. Let G = (C, L) bbeadirected graph, where each node in the set represents a network node with processing/queving and forwarding capa: bilities, and each oriented arc in L i a directional transmis- sion system (link). Each link has associated a cost measure defined by its physical properties and crossing tratfic low. [Network applications generate data flows from source to des- ‘ination nodes. For each node in the network, the local rout- ing component uses the local outing table to choose the best foulgoing link to direct incoming data towards their destina- tion nodes. The routing able R, = [ria] of a generic node 4, where N%; isthe set of neighbors of i, says 10 data pack- es entering node i and directed towards destination node which should be the next node 7 € JV, to move to. Routing tables are bi-dimensional because the choice ofthe neighbor node to which a data packet entering a generic node i should be forwarded isa function of the packet destination node d. ‘Aant-routing tables possess the same bi-dimensiona structure: ‘pheromone trails associated to each connection are vectors of ‘No = 1 values. In fact, ant-routing tables, in all the ACO implementations for routing, are used to build the routing ta- bles by means of implementation-dependent transformations. ‘These vectorial pheromone tails are the natural extension of the scalar trails used forthe TSP. ‘Other important differences withthe TSP implementation arise from the different nature of the two problems: (i) each ant is assigned a defined pair(s, d) of start-destination nodes and, discovering a path between s and d, the ant builds only @ “its e noted that inthe vga ant system (Derg, 1992 Daigo al, 1981) pheromone evaporation was performed before pberomove op ating. The algorithm presented hee aod the erga one ave exacly| teste i te values Ar*(0) wed i Equation 3 ae set © AT (0) = = 9) ISCO), part ofthe whole problem solution, defined in terms of paths between all the pairs (jj) i the network, (i) the cost asso- ciated tothe connections are not statically assigned: they de- ‘pend onthe connection’ physical properties and onthe taf fie erossing the connection, that interacts recursively with the routing decisions Inthe following subsection we presenta simplied version ofthe AmtNet algorithm, A full description of AntNet can be found in (Di Caro & Dorigo, 1998). 4a AntNet In AntNet, each ant searches for a minimum cost path be- tween a pair of nodes ofthe network. Ans ae launched from cach network node towards destination nodes randomly se- lected to match the traffic pattems. Each ant has a source node sand a destination node d, and moves fom # 0 d hop- ping from one node to the nex ill node dis reached. When ant k isin node i, it chooses the next node j to move to ac- cording to a probabilistic decision rule whichis a function of the at's memory JM and of the loal an-routing table A, ‘Pheromone trails are still connected to aes, but are mem- orized in variables associated to arc-destination pais. Thats, cach directed arc (i,) has No ~ 1 tril values 7a € [0,1] associated, one for each possible destination node d an ant located in node ican have (therefore, in general, re ¥ Ta) ach are has also associated an heuristic value mis € [0,1] independent ofthe final destination, The heuristic values are set to the following values _ te Liew where gi ithe Length (in bits waiting o be sent) of the «queue of the link connecting node i with is neighbor j. Ts AntNet, as wel asin most other implementations of ‘ACO algorithms for routing problems, the daemon compo: nent (line 6 of the ACO meta-heurstic of Figue 1) is not present “The local ant-outing table A, is obtained by a functional composition of the local pheromone trails ra and heuristic ‘values mj. While building the path tothe destination, ants ‘move using the same lnk queues as data. In this way ants expecience the same delays as data packets andthe time Tyg lapsed while moving from the source node sto the desta tion node d can be used as a measure ofthe path quality. The ‘overall "goodness" ofa path canbe evaluated by an heuristic function ofthe trip time Tyg and ofa local adaptive stats cal model maintained in each node. In fact, paths need to be evalated relative to the network status because a trip time T judged of low quality under low congestion conditions could bbe an excellent one under high traffic load. Once the generic ant has complete a path, it deposits onthe visited noses an amount of pheromone A7*(t) proportional tothe goodness ‘of the path it built. To this purpose, after reaching its dest nation node, the ant moves back tis souee nodes along the ny = ° 1474same path but backward and using high prosity queues, to allow a fast propagation of the collected information. During this backward path from d to the ant & increases the pheromone trail value ,(t) of each connection i, previ- ‘ously used while it was moving from s to d. The pheromone trail intensity is increased by applying the following rule: rugalt) nygalt) + Ark Ct) © ‘The reason the ant updates the pheromone tals during its backward trip is that, before ic ean compute the amount of| pheromone A7*(¢) to deposit onthe visited arcs it needs to ‘complete a path from source to destination to evaluat it ‘Alter the pheromone tail on the visited arc has been up- ated, the pheromone value of all the outgoing connections of the same node i, relative to the destination d, evaporates? alt) A) Gea EM where isthe se of neighbors of node i As we said, AntNe'santrouting table Ay = love) of node # is obtained, as usual, by the composition of the pheromone trail valves with the local heuristic values. This is done as follows: ® whore j © Ne i the destination node, [0,1] ‘sighting factor andthe denominator is normalization term ‘The ans decision rule is then defined as follows. Let, at sme fant be located on node i and be directed towards node do IF, AM, that if there is at least one city in the an’ curent location neighborhood that ant has not visited yet, then the ant chooses the next node j € Nj; with probability it sem halt) = { one oserwise, the ant chooses acty j AV with uniform prob- ability: p(t) = 1/((N4)) (Note hatin AntNet, diferealy from what happens in Ant System, the neighborhood and the feasible neighborhood are the same, that is, A =.) nother words, ans try to avoid eyeles (Equation 9) but, in the ease all the nodes ins neighborhood have already been visited by the ant, the ant as no choice and thas 10 re-visit anode, generating in this way a cycle. In this ease the generated cycle is deleted from the ant memory, that for gets completely sbout it. Considering the stochasticity ofthe decision rule and the evolution in the trafic conditions, its ‘very unlikely thatthe ant repeats the same cyele over and over sain, ausalt) 7 o 2m icize he ey fcr is chosen stat operates oemaliatin ‘ofthe pheromone values whic contin therefore tbe sb 5 probs 5 Other ACO meta-heuristic applications ‘There are now available numerous successful implementa- tions ofthe ACO meta-heuristic applied to a numberof dif ferent combinatorial optimization problems. "The most widely studied problems using ACO algorithms are the traveling salesman problem and the quadratic assign- ment problem (QAP) Results obtained by the application of ACO algorithms to the TSP are very encouraging (see (Stitzle & Dorigo, 19996) for an overview of applications of ACO algorithms to the ‘TSP): they are often better than those obtained using other general purpose heuristics like evolutionary computation or simulated annealing. Also, when adding to ACO algorithms rather unsophisticated local search procedures based on 3- opt (Lin, 1965). the quality of the results obtained (Dorigo & Gambardella, 997) is close to that obtainable by more so- phisticated methods. More research willbe necessary to 35 Sess whether ACO algorithms can reach the performance of state-of-the-art algorithms like lerated Lin-Kernighan (Jobn- son & McGeoch, 1997), AAs in the TSP cas, the fist application of an ACO al- gorithm tothe QAP has been that of Ant System (Maniez20 a, 1994). In the last two years several ACO algorithms forthe QAP have been presented by Maniezzo and Colomi (Maniez20 & Colom, 1999, in pres). Maniezzo (Maniezz0, 1998), and Stitae (Stitzle & Hoos, 19984). Currently, ACO algorithms are among the best heuristics for attacking real- life QAP instances. We refer to (Stele & Dorigo, 1999) for an overview ofthese approaches ‘The sequential ordering problem is closely elated t the asymmetric TSP, but additional precedence constraints be- tween the nodes have tobe stisted. Gambardellaand Dorigo have tacked this problem by an extension of ACS enhanced by a local search algorithm (Gambardella & Dorigo, 1997). ‘They obtained excellent results and were able to improve the best known solutions for many benchmark instances ‘first application of AS tothe job-shop scheduling prob- lem has been presented in (Colom er al, 1994). Despite showing the viability of the approsch. the computational resulls are not competitive with state-of-the-art algorithms, More recently, MMAS has been applied to the flow shop scheduling problem (FSP) in (Stile, 1998). The computa- tional resus have shown that \M.MAS outperforms earlier ‘proposed Simulated Annealing algorithms, while it performs comparably to Tabu Search, Another very recent applica- tion is that tothe single machine total tardiness problem, dis- cussed in a paper published in these same proceedings (Bauer etal, 1999). Again, the results obtained are very promising Costa and Herz (Costa & Hertz, 1997) have proposed an extension of AS to assignment ype problems and present an application of their approach to the graph coloring prob- Sanoties very gpd algorithm inspired by the ant colony meapor i HAS QAP (Gambari ea, 1997) 1s ot sted wit the oes above esas dee follow the ACO ent beuii larslem obtaining results comparable to those obtained by other heuristic approaches ‘An application Of ASyaq¢ 10 the capacitated vehicle routing problem is presented by Bullnheimer, Hart, and Strauss (Bullnheimer er al, 1997). They obtained good computational results for standard benchmark instances, slightly worse than the best performing Tabu Search algo- rithms. A recent application by Gambardella,Taillard, and ‘Agazzi to vehicle routing problems with time windows im- proves the best known solutions for some benchmark in- stances (Gambardella eta, 1999). Michel and Middendorf (Michel & Middendorf, 1998; Michel é& Middendorf, 1999) have proposed an application ‘of AS tothe shortest common supersequence. They introduce the use of a lookahead function during the solution construc- tion by the ants. They also presenta parallel version oftheir algorithm based on the island model typical of many parallel zgenecc algorithms implementations. AMMAS has recently been applied to the generalized assignment problem by Ramalhinho Lourengou and Serra (Lourengo & Serra, 1998), obtaining very promising com- putational results. In particular, their algorithm is shown to find optimal and near optimal solutions faster than a GRASP algorithm which was used for comparison, ‘Another very recent application of the ACO mete-heuristic is the one to the multiple knapsack problem proposed in fone of the articles published in these same proceedings (Leguizamén & Michalewicz, 1999). Also in this ease the obtained results are very promising Perhaps the currently most active research area in ACO algorithms is the study of their application to telecommu- nication network problems, in particular to network routing (Schoonderwoerd er al, 1996; Di Caro & Dorigo, 1998). ‘The application of ACO algorithms to network optimization problems is appealing because these problems have character istics lke distributed information, non-stationary stochastic dynamics, and asynchronous evolution ofthe network status ‘which well match some of the properties of ACO algorithms like the use of local information to generate solutions, indi- rect communication via the pheromone trails and stochastic sate transitions. A detailed overview of routing applications ‘of ACO algorithms can be found in (Dorigo et a, 1998). 6 Conclusions In this paper we have given a formal description of the Ant Colony Optimization meta-heuristic, as well as of the class of problems to which it can be applied. We have then described ‘wo paradigmatic applications of ACO algorithms tothe trav- cling salesman problem and to routing in packet-switched networks, and concluded with a short overview on the cur- rently available applications. Ongoing work follows three ‘main directions: the study of the formal properties ofa sim- plified version of ant system, the development of AntNet for Quality of Service applications, and the development of fur- ther applications to combinatorial optimization problems. 7 Acknowledgments Marco Dorigo is a Research Associate ofthe Belgian FNRS. Gianni Di Caro was supported by a Madame Curie Fellow: ship (CEC-TMR Contract N. ERBFMBICT 961 153). References A. Bauer, B. Bullnheimer, RF. Hart, & C. Strauss. 1999. ‘An Ant Colony Optimization Approach for the Single Machine Total Tardiness Problem. In: Proc. of 1999 Congress on Evolutionary Computation. IEEE Press. Bertsekas. 1995. Dynamic Programming and Optimal Control. Athena Scientific. Bullnheimer, RF Hart, & C. Strauss. 19974. An Im- proved Ant System Algorithm for the Vehicle Routing Problem. Tech, rept. POM-10/97. Insitute of Manage: ‘ment Science, University of Vienna, Austria, Accepted for publication inthe Annals of Operations Research Bullnheimer, R. F. Hal, & C. Strauss. 19976. A New Rank-Based Version of the Ant System: A Computa- tional Study. Tech, rept. POM-03/97. Insitute of Man- ‘agement Science, University of Vienna, Austria. Ac- cepted for publication in the Central European Journal {for Operations Research and Economics. Colorni, M. Dorigo, V. Maniezzo, & M. Trubian. 1994. ‘Ant System for Job-Shop Scheduling. Belgian Journal of Operations Research, Statistics and Computer Sci- ence (JORBEL), 34, 39-53, D. Costa, & A. Hertz, 1997. Ants Can Colour Graphs. Jour- nal ofthe Operational Research Society, 48, 295-305. G. Di Caro, & M. Dorigo. 1998. AntNet: Distributed Stig- smergetie Control for Communications Networks. Jour. nal of Artificial Intelligence Research (JAIR), 9, 317~ 368. G.DiCaro, & M. Dorigo. 1998. Two Ant Colony Algorithms {for Best-Effort Routing in Datagram Networks. Pages 541-546 oft Proceedings ofthe Tenth IASTED Interna tional Conference on Parallel and Distributed Comput ing and Systems (PDCS'98). [ASTEDIACTA Press. Dorigo. 1992. Optimization, Learning and Natural Algo rithms (in Ualian). Ph.D. thesis, Dipartimento di Ele ‘ronica, Politecnico di Milano, Italy. Dorigo, & L. M, Gambardella. 1997. Ant Colony Sys- tem: A Cooperative Learning Approach tothe Traveling Salesman Problem. IEEE Transactions on Eveluronary ‘Computation, \(1), 33-66. Dorigo, G. Di Caro, & L. M. Gambardella. 1998. ‘Ant Algorithms for Discrete Optimization. Tech. rept 1476IRIDIAB-10. Université Libre de Bruxelles, Belgium. ‘Accepted for publication in Artificial Lye M. Dorigo, V. Maniezz0, & A. Colomi. 1991. Pastive feed- back as a search strategy. Tech. rept. 91-016. Diparti- ‘mento di Elettronica, Politecnico di Milano, Italy M. Dorigo, V. Maniezz0, & A. Color. 1996. The Ant Sys- tem: Optimization by a colony of cooperating agents IEEE Transactions on Systems, Man, and Cybernetics Part B, 26(1), 29-41, LM. Gambardella, & M. Dorigo. 1997. HAS-SOP: An Hy: brid Ant System for the Sequential Ordering Problem. ‘Tech. rept. IDSIA-11-97. IDSIA, Lugano, Switzerland, L. M, Gambardella, E. D. Tallard, & M. Dorigo. 1997. Ant Colonies forthe QAP. Tech. rept. IDSIA-4-97. IDSIA, Lugano, Switzerland. Accepted for publication in the Journal ofthe Operational Research Society (JORS). L. M. Gambardella, E. D. Taillard, & G. Agazzi. 1998. 'MACS-VRPTW: A Multiple Ant Colony System for Ve- hele Routing Problems with Time Windows. In: D. Corne, M. Dorigo, & F. Glover (eds), New Ideas in Op- timization. MeGraw-Hill MR. Garey, & D. S. Johnson. 1979. Computers and In ‘tractablity. WH. Freeman and Company. D.S. Johnson, & L.A. MeGeoch, 1997. The Traveling Sales: man Problem: A Case Study. Pages 215-310 of: E-H. ‘Aarts, & JK, Lenstra (eds), In Local Search in Combi- ‘narorial Optimization. Chichester: John Wiley & Sons EL. Lawler, J. K.Lensira, A. H. G. Rinnooy-Kan, & D. B. ‘Shmoys (eds). 1985. The Travelling Salesman Problem. Wiley. G. Leguizamn, & Z. Michalewicz. 1999. A New Version ‘of Ant System for Subset Problems. In: Proc. of 1999 Congress on Evolutionary Computation. IEEE Pres. 'S. Lin, 1965, Computer Solutions of the Traveling Salesman Problem. Bell Systems Journal, 44, 2245-2260, H. Ramalhinho Lourengo, & D. Serra. 1998 (May). Adaptive ‘Approach Heuristics for the Generalized Assignment Problem. Economic Working Papers Series 304, Uni verstat Pompeu Fabra, Dept. of Economics and Man- agement, Barcelona, Spain. V. Maniezzo. 1998. Exact and Approximate Nondetermin- istic Tre Search Procedures for the Quadratic Assign- ‘ment Problem. Tech. rept. CSR 98-1. C. L. in Scienze ell Informazione, Université di Bologna, Italy. \V. Maniezzo, & A. Color. 1999, in press. The Ant System Applied to the Quadratic Assignment Problem. IEEE ‘Trans. Knowledge and Data Engineering. V. Maniezzo, A. Color, & M. Dorigo. 1994. The Ant System Applied 10 the Quadratic Assignment Problem. ‘Tec. rept. IRIDIAV94-28. Université Libre de Bruxelles, Bel- ium. Michel, & M. Middendorf. 1998. An Island Model based ‘Ant System with Lookahead for the Shortest Superse- quence Problem. Pages 692-701 of. A. E. Eiben, T. Back, M. Schoenauer, & H-P. Schwefel (eds), Proceed: ings of PPSN-Y, Fifth International Conference on Par- allel Problem Solving from Nature. Springer-Verlag. Michel, & M. Middendorf. 1999. An ACO Algorithm for the Shortest Common Supersequence Problem. In: D. ‘Come, M. Dorigo, & F. Glover (eds), New Ideas in Op- ‘mization. MeGraw Hil Reinelt. 1994. The Traveling Salesman Problem: Con putational Solutions for TSP Applications, Bertin: Springer-Verlog ‘Schoonderwoerd, O. Holland, J. Bruten, & L. Rothkrantz 1996. Ant-based Load Balancing in Telecommunica: tions Networks. Adaprive Behavior, $(2), 169-207. Stitzle. 1998. Parallelization Strategies for Ant Colony Optimization. Pages 722-731 of- A. E.Fiben, T. Back, M. Schoenauer, & HP. Schwefel (eds), Proceedings of PPSN-V, Fifth International Conference on Parallel Problem Solving from Nature. Springer-Verlag ‘ttle, & M. Dorigo. 19992. ACO Algorithms for the Quadratic Assignment Problem. In: D. Come, M. Dorigo, & F. Glover (eds), New Ideas in Optimization. McGraw-Hill ‘Stitzle, & M. Dorigo. 1999. ACO Algorithms for the ‘Traveling Salesman Problem. In: K. Miettinen, MM. Mikeli, P. Neittaanmiki, & J. Periaux (eds), Bvolu tionary Algorithms in Engineering and Computer Sci- ence: Recent Advances in Genetic Algorithms, Evo- lution Strategies, Evolutionary Programming, Genetic Programming and Industrial Applications, John Wiley & Sons. ‘Stitzle, & H. Hoos. 1998a. MAX-MIN' Ant System and Local Search for Combinatorial Optimization Prob- lems. Pages 313-329 of S, Voth S. Martello, LH. Os- ‘man, & C. Rouesirol (eds), Meta-Heurstcs: Advances ‘and Trends in Local Search Paradigms for Optimization. Kluwer Academics, Boston Sule, & Holger Hoos. 1998b. Improvements on the ‘Ant System: Introducing the MAX-MIN Ant Sys- tem. Pages 245-249 of RF. Albrecht G.D. Smith, NC. Steele (ed), Artificial Newral Networks and Genetic Algorithms. Springet Verlag, Wien New York. unr