Ant Colony Optimization Israel Beniaminy Agenda

Ant Colony Optimization Israel Beniaminy

Agenda • • What ants can do How real ants succeed Artificial ant algorithms Ant Colony Optimization ACO in TSP problems Further applications The role of local search

What ant colonies can do • • Build complex structures Gather food Take care of their young Herd other species Form bridges made of live ants Fight wars with other colonies Ant colonies have their own “personality”, persisting beyond individual ant’s life time

Bridge of Ants

Ants farming aphids for honey

Ants transporting heavy objects

What individual ants can do • Sense – Sight (typically limited) – Smell and touch (food, objects, other ants) • • Move Carry Use feet and jaws (dig, attack) Deposit and sense pheromones How can complex ant-colony behavior emerge from these simple behaviors ?

Emergence of ordered behavior • Navigating: Ant trail • Foraging: Leafcutter ants collecting petals

Two ways to create ordered behavior • Controlled top-down • Emerging via micro-interactions לך-אל-נמלה עצל; ראה דרכיה . וחכם אשר אין-לה קצין-- שטר . ומשל

Stigmergy: One of the ants’ secrets • Pheromone: substance released by one individual that modifies the behavior of another of the same species Greek: Pheran-transfer, Horman-excite. • Stigmergy: Co-operative interaction through indirect modification and sensing of the environment Defined by entomologist Pierre Grassé (1959). Greek: Stigma-sign, Ergos-work.

Example of ordered behavior: Raiding Army Ants • An example of powerful, totally decentralized control. • Example : Eciton burchelli (army ant) swarms can consist of as many as 200, 000 workers. • These individuals communicate and coordinate via pheromones.

Eciton burchelli = The Army Ant

Using stigmergy to find paths Ants secrete pheromones while traveling from the nest to food, and vice versa in order to communicate with one another to find the shortest path.

Using stigmergy to “fix” paths

History of Ant Algorithms • Goss et al. 1989, Deneuborg et al. 1990, experiments with Argentine ants • Dorigo et al. 1991, applications to shortest path problems • Now: established method for various optimization problems

How real ants solve Shortest Path • Ants choose paths depending on pheromone • After collecting food, paths are marked • After some time, the shortest path has the highest probability

Ant Colony Optimization • Artificial ants form a multi-agent system performing the functions as observed in the real ant system • Exploit stigmergistic communication The ACO meta-heuristic relies on the cooperation of a group of artificial ants to obtain a good solution to a discrete optimization problem such as the TSP

Traveling Salesperson Problem • Brief reminder: – Sales person needs to find a tour of N cities that has the minimum total travel time – For many problem sizes, no practical way is known to find the best (optimal) tour • Highly interesting problem – For computer-science theory – In solving real-life situations

ACO for TSP START ACO Locate an ant randomly in a city, and store the current city in a tabu list Determine probabilistically as to which city to visit next Move to next city and place this city in the tabu list NO NO Have all cities been visited? Have K Iterations been performed? Update Pheromone: Add according to tour length, and evaporate YES Record the length of tour and clear tabu list Repeat for the M ants in the colony YES STOP ACO

Key Parameters q Trail intensity is given by value of ij which indicates the intensity of the pheromone on the trail segment, (ij) q Trail visibility is ij = 1/dij : a heuristic function of the desirability of adding edge (dij is distance from i to j) q The importance of the intensity in the probabilistic transition is q The importance of the visibility of the trail segment is q The trail persistence or evaporation rate is given as q Q is a constant and the amount of pheromone laid on a trail segment employed by an ant q Initial pheromone is a small amount on all edges

Probabilistic City Selection q Each ant iteratively builds a tour by selecting the next city to visit, using this probability q Determined by ij - pheromone content in an edge (ij), ij - trail visibility, and , : their importance Probability for ant k to select edge (ij) Jk(i) is the set of cities not visited by ant k before step i

Pheromone Updating For Each Ant q Tk(t) is the tour found by k-th ant, and its length is Lk. Pheromone increment to each edge of this tour is: q Update pheromone using this increment, but also evaporate some pheromone:

Example ACO for TSP AIDemo. exe

Variations and Challenges § Balancing Exploration and Exploitation § Exploration: Trying new directions, even when the heuristics and current pheromone level indicate they aren’t good Avoid being stuck in local optima § Exploitation: “Believing” the heuristics and pheromone trail and following it with high probability Find a good solution quickly § When to update? § All ants? § Best ant(s)? Note: Some variations introduce global, centralized control, unlike “real” ants § In this colony? Across all past colonies? § Remove pheromones from trails used by worst ants? § How many ants?

Time-behavior of solution quality Improvements slow down as the algorithm progresses:

Time-behavior of solution quality (2) This is roughly a log-linear relationship: Could we expect better behavior?

Further Applications § TSP: good, but better methods exist § QAP: best available heuristic for structured problems § Vehicle routing: among the best methods for routing with time window constraints § Sequential ordering: best available heuristic § Graph colouring: good, but not the best § Shortest common super-sequence: among the best

Optimizing VRPTW Vehicle Routing Problem with Time Windows: § Several vehicles § Each “city” has a time window - it should be visited within it § Vehicle capacity limitations § Minimize travel and/or number of vehicles

Optimizing VRPTW

Introducing Local Search into ACO § After an ant completes building a solution, try to improve it using a set of simple operators § Search perspective: Looking for a nearby “hill top” § Informally, this is smoothing the rough edges of the current solution § The improved solution is used for pheromone update Solution quality Iteratively built solution Solution improved by local search

Example local search operator: Insert

Example local search operator: Replace

Problems with ACO § Like any search process, ACO can also get stuck in local optima § Options when change has stopped: § Restart – re-initialize all the pheromones § Increase exploration, decrease exploitation § Change the update scheme: reduce effect of best ants and let “not-so-best” ants change pheromones § Find parts of the solution that seem to be improvable, and: § Change pheromones for those parts, or … § Temporarily change value function to focus change directions

Further Reading § Web: § Computer Science: http: //iridia. ulb. ac. be/~mdorigo/ACO § Myrmecology (the scientific study of ants): http: //www. myrmecology. info/index 2. html § Books (Computer Science): § Ant Colony Optimization, Dorigo and Stützle. MIT Press, 2004 § Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, 1999.

Thank you