You are on page 1of 9

CS 580 Final Project Kevin Workman December 14th, 2011

Usable Emergence in a Path-Finding Ant Colony Simulation

In nature, ants use pheromones to communicate with each other, allowing relatively simple behaviors performed by single ants to combine and result in complicated actions exhibited by the colony as a whole. The emergence of these complicated behaviors has been applied to the computer science world, and this project aims to make these principles available to both novice and experienced users through the creation of a usable interface for interacting with an ant colony simulation environment.

Ants are relatively simple insects that by themselves demonstrate very few behaviors that most humans would call intelligent, instead behaving reflexively based on their immediate environment. However, when enough ants get together, a colony can exhibit complicated behaviors required for survival in a constantly changing environment that contains food as well as obstacles and kids with magnifying glasses. One way they accomplish this is through communication via pheromones, which an ant leaves on the ground to give its comrades information about the environment. By simply responding to the pheromones in its immediate environment and leaving pheromones of its own behind, a single ant becomes a small piece of a more complicated whole. Without having any knowledge of the higher-level facts about their environment, such as how to navigate from the nest to a food source and back again, ants are able to do exactly that and thrive, thanks to these emergent behaviors. The idea of allowing complicated behaviors to emerge from the interaction of relatively simple agents has been applied to the computer science concept of search. Agents must navigate unknown and potentially blocked or dangerous environments in order to accomplish path-finding goals. Instead of investing vast resources in creating more advanced and intelligent individual agents that know or can learn everything about an environment, this area of study goes the opposite direction and uses entire colonies of simple agents that exhibit behaviors similar to those of ants.

Since the early 1990s, the idea of applying inspiration gained from ant colony behavior has successfully been used to navigate open environments that mirror the properties of the natural world as well as more classic computer science search problems such as the travelling salesperson problem. The algorithm takes inspiration from the way real-world ants navigate their environment in order to find food. Although each ant is relatively simple, the basic behaviors of many ants combine and interact to form the more complex behaviors of the total ant colony. The colony as a whole accomplishes complex tasks such as finding food while responding to dynamic and dangerous environments, even though at the ground level, individual ants are not doing anything more complicated than reflexively responding to immediate stimuli. This principle shows that complicated behaviors can emerge from simplistic rules mingling with each other, and it can be seen in everything from John Conways Game of Life (Gardner 1970) to the human brain, where relatively simple individual brain cells interact to result in our own complicated behavior. Philosophy aside, the ant colony optimization algorithm uses these principles in order to accomplish tasks such as path planning, graph traversal, and even the travelling salesperson problem. The basic algorithm is as follows: 1- Ants originate from the nest cell and travel to neighboring cells randomly. 2- When an ant comes across a food cell, it returns to the nest, leaving a pheromone behind on every cell through which it passes. 3- As ants discover a cell containing a pheromone trail, they are more likely to follow it and find the food. 4- Pheromones evaporate over time. Longer paths require more traversal time, so they deteriorate more rapidly than shorter paths. 5- As longer paths deteriorate, shorter paths are preferred, and the ants converge on the shortest path from the nest to the food. The algorithm itself has gone from a single-pheromone model as described above laid out by Marco Doringo in 1992 (Dorigo, 1992) to a weighted, double-pheromone model implemented by Liviu Panait and Sean Luke (Panait and Luke, 2004), which this project more closely resembles and is described below. There has also been work in applying algorithmic learning to the process (Panait and Luke, 2004), which could be explored as an extension to this project using the parameter adjustment feature, also described below. As more robotic swarms using these principles are developed (Labella, Dorigo, Deneubourg 2006), I predict algorithms like this one will gain more mainstream notoriety. Therefore, part of my goal was to take this concept, which is simultaneously simple and advanced, and make it more available to people outside of the artificial intelligence or computer science field as well as to veterans of the AI world.

Development Process
The MASON ant colony simulation ( very successfully demonstrates the principles of ant colony search in a static environment, and access to the code gives intermediate and experienced users the ability to experiment with the simulation. However, it is not easy for a novice user with little or no programming experience to make changes to the simulation. Even with coding experience, it is not possible to dynamically modify the simulation environment, for example by adding or removing obstacles or food while the simulation is still running. These issues cut down on the usability (ISO 9241) of MASON, especially for novice users.

Figure 1: MASON ant colony found a path from nest to food but now what?

With those issues in mind, I started creating a simulation environment from the ground up, based on the properties described in class as I understood them. The foundation of the program is a 2-dimensional grid of adjustable size (50 by 50 cells is the default value), where each grid cell represents a discrete location in the environment. Cells can contain any number of ants, as well as a nest or a food cell. Additionally, any cell can contain an obstacle, which blocks ant agents from that cell, and each cell also contains a pheromone value. In my original attempt, an ant agent travelled randomly until it found food, and then it retraced its path exactly, leaving behind pheromones that alerted other ants to follow the same path to the food. However, several ants would find multiple paths to the same food, which resulted in mostly random behavior and the entire environment filling up with pheromones. Obviously, this model was too simplistic.

I then modified my work to include a second pheromone, left behind by the ants as they searched for food. This pheromone was followed by ants searching for the nest after finding food. However, multiple ant agents found different paths to the food, which again caused mostly random trails to be followed as ants searching for food were following ants searching for the nest, who were following the ants searching for the food! I also tried weighting the amount of pheromone left behind by each ant based on how long the ant has been alive to give less focus to ants wandering aimlessly, but the ants still mostly wandered in circles. Feeling frustrated and discouraged, I took a closer look at MASON and the papers it references. I discovered that the MASON ant colony simulation caps the two pheromones at the value of that cells neighbors. This was a bit of a surprise to me, as it seemed to violate the concept that an individual ant knows nothing of its environment outside of its current location. However, it seems reasonable to include a locations neighbors in the ant agents immediate environment. Adding this behavior drastically increased the effectiveness of the simulation. Instead of wandering randomly, ants quickly converged on a path from the nest to the nearest food. However, instead of MASONs default best-neighbor approach where ants always choose the neighboring cell with the highest pheromone level, I opted for an approach where ants choose the next cell based on the weighted value of every cells pheromone level. From a cells unobstructed neighbors, an ant chooses the next cell to traverse by totaling up the neighboring cells pheromones and using that as the denominator and the pheromone level of each individual cell as the numerator, to find the chance that each cell should be chosen. Cells without pheromones are given a base value so that ants always have a chance to explore. As a simple example, say an ant is currently located on a cell with two unobstructed neighbors, cell A and cell B. Cell A has a pheromone level of 75, and cell B has a pheromone level of 25. That means that the ant has a 75% chance of choosing cell A. In practice, the numbers are almost never that simple, but this seemed more realistic to me, and it allowed the simulation to respond to dynamic changes in the environment by allowing occasional exploration. This effect is similar to increasing the chance of random action in MASON, but it is more directed than random action. Once the groundwork was laid out, I gave the user the ability to manipulate the simulation by dynamically changing the environment. This was accomplished through a straightforward user interface- the user can select a type of tile to place, and clicking within the simulation adds that tile. Available tiles are obstacle (ants cannot enter that tile), nest (ants originate from that tile), food (ants must enter that tile before returning to the nest), or clear (nothing is in that tile). By adding or removing different kinds of tiles, the user can easily experiment with the reaction of the ant colony simulation to dynamic environments.

Figure 2: Ants adapting to a dynamic environment, snapshots taken throughout a single run. Food is found, an obstacle is added and circumnavigated, and a closer food is added, discovered, and eventually preferred.

My next step was to parameterize the simulation, allowing the user to customize the ant behavior as well as the environment. The parameters are the maximum pheromone level, the dropoff rate, and the evaporation rate. The maximum pheromone level controls the pheromone value placed by an ant into a cell containing food or a nest. Ants are more likely to choose tiles containing higher pheromones, but lower pheromone levels encourage exploration of the environment as the ant agents are relatively more likely to choose cells with little or no pheromones. The dropoff rate controls the pheromone limit of any particular cell- a cells pheromone level is set to that of its highest neighbor, multiplied by the dropoff rate. This causes cells closer to food or nests to contain higher pheromone levels than cells that are further away, which the ants use to find their way between nests and food. The evaporation rate controls how rapidly pheromones degrade, which causes older pheromone paths to be less desirable. By manipulating these parameters, users can explore the effect of each as well as find the best combination for different scenarios. Other, more obvious parameters are the speed of the simulation, the number of ants in the simulation, and the size of the environment. The speed of the simulation simply controls the pause between steps, which allows a user to either examine each step or fast forward the simulation. The number of ants in the simulation can allow a user to either observe the behavior of a single ant or use many ants to explore the environment more fully. I also added the option to toggle ant behavior between the default weighted neighbor behavior and an option to always choose the neighboring cell with the highest pheromone level, much like the MASON ant colony simulation. As described above, the weighted behavior assigns each neighboring cell a probability to be chosen, which encourages exploration of the environment. The highest pheromone behavior always chooses the cell with the highest pheromone level, which encourages the ants to converge on the shortest path more rapidly. The interactive simulation environment and access to the simulation parameters give users the ability to explore the principles of emergence in an ant colony simulation. By placing importance on ease of use, the program is available to novice users- a novice user can begin interacting with the environment and playing with the ideas of emergence without delving into the details of the algorithm. And by making the simulation parameters available, the program allows more experienced users the option of more fully experimenting with the simulation.

Extra Work
While the above process accomplished my goal of developing a usable ant colony simulation with an interactive environment that encouraged both novice and experienced users to explore the principles of emergence, I took advantage of the extended submission deadline to improve the program further. As a replacement to the option to toggle between the weighted neighbor behavior and the best neighbor behavior, I implemented an option to mix the two in a more analog way. By adjusting a slider, the user can specify the likeliness that an ant chooses the bestneighbor over the weighted neighbor approach. If the slider is all the way to the left, ants always follow the weighted behavior, and if the slider is all the way to the right, ants always follow the best neighbor behavior. If the slider is halfway in the middle, ants will choose one approach half the time, and the other approach the other half of the time. This allows the user to come up with a hybrid behavior between my original weighted idea and the default MASON algorithm, which helps the user find the best behavior for each environment as well as simply compare the two approaches. One of my subsequent goals was to simulate a travelling salesperson optimization problem using the ant colony algorithm to produce reasonable result paths. My original plan was to use obstacles to narrow down the simulations path options to obvious routes between food cells, so that the ants could use a single pheromone to find the shortest path between multiple food cells. However, I instead came up with a system that allowed distinct pheromones for each individual food cell. This idea, coupled with an option to require ants find every food cell before returning to the nest, made it possible to simulate the travelling salesperson problem. Although the paths the ant colony simulation comes up with are not usually optimal, they are reasonable solutions to the travelling salesperson problem.

Figure 3: Ants coming up with a reasonable solution to a TSP problem with 25 goals.

The multiple-pheromone algorithm works very similarly to the two-pheromone approach. However, instead of containing just a nest pheromone and a food pheromone, each cell can contain a unique pheromone for each food cell in the simulation. Ants can choose between cells based on all of the pheromones- either by weighing each of them against the total neighboring pheromone levels, or by simply choosing the highest pheromone

level, or even a hybrid of both using the slider described above. For simplicitys sake, the highest pheromone value for each cell is displayed in the simulation, but they are all still represented by the underlying model. In addition to the travelling salesperson environment, the simulation parameters allow the user to tweak the algorithm to produce reasonable results for most environments. However, to react to dynamic changes in the environment, the user oftentimes had to make manual changes to the parameters to encourage the ant agents to come up with satisfactory paths. For example, small environments with only a single piece of food are more successfully solved with high pheromones, low evaporation, and low dropoff levels. This combination encourages the ants to converge on a path rapidly. As more food or obstacles are added, or the simulation size increases, the pheromone level should decrease and the evaporation and dropoff levels should increase to encourage exploration. Because the focus of the project was usability, requiring the user to tweak parameters every time a change is made violates the goal of ease of use. Therefore, I implemented a feature that automatically adjusted the parameters over time. This feature reacts to changes in the environment, increasing and decreasing each parameter over time in an effort to find a suitable combination of parameters to changing environments. The deltas of each parameter, which affect the rate at which each parameter is adjusted, are themselves adjustable, allowing advanced users to tweak the automatic adjustment of parameters. Also, the length of time the parameter adaptations take to complete is adjustable, and this entire feature can be disabled to allow users to make manual adjustments.

The result of my work is an interactive program that allows novice and experienced users to explore the principles of emergence by focusing on usability. The program encourages interaction by making it easy to make changes to the environment on the fly, and the simulation reacts appropriately by either allowing the user to adjust the parameters or by adjusting the parameters automatically. This program is available as an applet, a webstart, and a runnable jar at []. The source code of the program, as well as this paper, is also available on that page.

Figure 4: Ants navigating a complex environment.

Conclusions and Future Work

The program accomplishes all of my initial goals, as well as a few goals that sprang up during the development process. It serves as an introduction to the principles of emergence in an ant colony simulation by encouraging interaction with the environment, and it exposes some of the more advanced facets of the environment by allowing the user to adjust the parameters of the simulation. The source code might help intermediate programmers as a base for continued development. I hope that the project serves as an example of some of the less obvious facets of the artificial intelligence field and that it might inspire novice programmers to investigate further. That being said, the program would benefit greatly from optimization, as the program does tend to lag quite a bit, especially when many different food cells are active. This optimization could take the form of using multiple threads, or simply eliminating extraneous calculations. The current program is less than optimal, as features such as using multiple food pheromones and updating parameters on the fly were not in the original program design. Additionally, the default and automatic parameters I came up with are not guaranteed to find the optimal path. The system usually comes up with a reasonable solution, but further experimentation with the parameters is required in order to find the optimal solution to any particular problem. The environment might be tweaked to include more realistic distance calculations. An easy example is the fact that moving to a diagonally adjacent cell should be a slightly longer distance than moving to an immediately adjacent cell, but in fact the simulation treats them each as taking exactly one simulation step to complete. This leads to zigzagging paths to be counted as the same distance as a straight path, which is obviously not optimal. Although the simulation can be used to model a lenient version of the travelling salesperson program, it would be interesting to model a stricter version. Currently, ants are able to go through a goal more than once, which violates the rules behind the travelling salesperson problem. This rule was not enforced in the current version, as the current ant behavior is more true to the original goal of setting up an emergent foraging environment, and the results are still reasonable and quite interesting.

Gardner, Martin (1970-10). Mathematical Games - The fantastic combinations of John Conway's new solitaire game "life". 223. pp. 120123. ISBN 0894540017. ISO (International Organization for Standardization). (1998). ISO 9241-11: Ergonomic requirements for office work with visual display terminals (VDTs): Guidance on usability. M. Dorigo (1992). Optimization, Learning and Natural Algorithms, PhD thesis, Politecnico di Milano, Italie. Panait, Liviu and Luke, Sean (2004). Ant Foraging Revisited. In ALIFE 9. Panait, Liviu and Luke, Sean (2004). Learning Ant Foraging Behaviors. In ALIFE 9. Labella T.H., Dorigo M., Deneubourg J. (2006). Division of Labour in a Group of Robots Inspired by Ants' Foraging Behaviour. ACM Transactions on Autonomous and Adaptive Systems 1(1):4-25.

Online Appendix The program is available as an applet, a webstart, and a runnable jar file at the above web address. The source code, as well as this paper, is also available there.