Multi-swarm Problem Solving in Networks Tony White email

Multi-swarm Problem Solving in Networks Tony White email: tony@sce. carleton. ca

Overview • Introduction – What is it and why is it interesting? – Problem solving and stigmergy. • Swarm problem solving – Swarm agent architecture – Connection-oriented routing – Simple diagnosis • Extensions and Future

What is Swarm Intelligence? • “Swarm Intelligence is a property of systems of non-intelligent robots exhibiting collectively intelligent behavior. ” [Beni, 89] • Characteristics of a swarm: – distributed, no central control or data source; – no (explicit) model of the environment; – perception of environment, i. e. sensing; – ability to change environment.

What is Swarm Intelligence (cont. )? • Swarm systems are examples of behaviorbased systems exhibiting: – multiple lower level competencies; – situated in environment; – limited time to act; – autonomous with no explicit control provided; – problem solving is emergent behavior; – strong emphasis on reaction and adaptation;

Motivations • Robust nature of animal problem-solving – simple creatures exhibit complex behavior; – behavior modified by dynamic environment. • Emergent behavior observed in: – bacteria – ants – bees –. . .

Emergent Problem Solving • For Lasius Niger ants, [Franks, 89] observed: – regulation of 1 degree celcius range; – forming bridges; – raiding specific areas for food; – building and protecting nest; – sorting brood and food items; – cooperating in carrying large items; – emigration of a colony; – finding shortest route from nest to food source; – preferentially exploiting the richest food source available.

Stigmergy • Indirect communication via interaction with environment [Grassé, 59] – Sematonic [Wilson, 75] stigmergy • action of agent directly related to problem solving and affects behavior of other agents. – Sign-based stigmergy • action of agent affects environment not directly related to problem solving activity.

Ant Colony • Ants are behaviorally unsophisticated; collectively perform complex tasks. • Ants have highly developed sophisticated sign-based stigmergy – communicate using pheromones; – trails are laid that can be followed by other ants.

Pheromone Trails • Species lay pheromone trails travelling from nest, to nest or possibly in both directions. • pheromones evaporate. • pheromones accumulate with multiple ants using path. Nest Food source

Pheromone Trails continued E E E T=1 T=0 30 ants D d=1. 0 d=0. 5 H C d=1. 0 B A 15 ants D 15 ants C H d=0. 5 15 ants 10 ants B 15 ants 30 ants A D 20 ants C H 10 ants B 20 ants 30 ants A

Swarm operation • Swarm agents: – arrive at a node, – sense environment, – undertake local activity, – modify environment, – use sensory input to make migration decision

Swarm Agent Architecture A=(E, R, C, MDF, m) • Agents have a uniform architecture consisting of five components: · emitters (E), · receptors (R), · chemistry (C), · a migration decision function (MDF), · memory (m)

Multi-swarm Architecture

Emitters • Generators of chemical messages (E) Chemical modification of environment

Receptors • Sensors of chemical messages (R) from local environment Detector for local environment

Chemicals • Chemicals are digitally-encoded using a {1, 0, #} alphabet. • The # symbol unifies with 1 or 0. • Chemicals have two attributes: – encoding – concentration • Chemicals participate in reactions.

Chemistry • The set of chemical reactions (C) that can operate on sensed and locally stored chemicals. Chemical interactions

Examples of chemical reactions Catalytic breakdown of 011 Endothermic reactions Exothermic reactions

Migration Decision Function • The MDF is used to determine the next node to visit in the network pijk (t) = Pp[Tijkp(t) ]-akp[C(i, j)]-b / Nk(i, j, t) = Sj in A(i) Pp [Tijkp(t) ]- akp[C(i, j)]-b akp, b are control parameters Nk(i, j, t) is a normalization term A(i) is the set of available egress links

Multi-Swarm scenario • Distributed Network Management employing delegation [Yemini, 91] • Three interacting swarms: – connection finding, – connection monitoring, D – connection fault diagnosis A E B C

Routing Problem • Idea – Ants dropping different pheromones used to compute “shortest” path from source to destination(s); – more flexible adaptation to failures and network congestion; – use only local knowledge for routing and avoid costly communication of state to all network nodes.

Why Routing? • Conventional routing often relies on: – global state available at all nodes; – centralized control; – fixed “shortest path” (Dijkstra) algorithms; – limited ability to deal with congestion or failure. • Ideally, would like to have network adapt routing patterns to take advantage of free resources and move existing traffic if possible.

Routing Research • Three approaches so far investigated: – [White et al, 96+] – [Schoonderwoerd et al, 97] (*) – [Di Caro and Dorigo, 97] • Differences – Link cost metric constant in * – Point to point traffic only in * – AS parameter settings constant in *

Routing agents • Agent types: – explorer • used for route determination – allocator • allocates resources in network when route emerged – deallocator • deallocates resources in network at end of call

Point-2 -Point Connections • For explorer agents: – At each node, they choose path with probability proportional to f(ce, pe); – explorers visit edges once only (achieved through use of tabu list); – when destination reached, ants return along the path explored laying down pheromone trail; – when explorers return a decision is made regarding path emergence

Path Emergence • At source node (“nest”): – store paths for previous m explorer agents; – when p% follow same path allocator agent is sent to allocate bandwidth in network; – explorer agents continue to look for new (possibly better) paths. • Applies for one or many pt-2 -pt connections: – ants use different, non-reacting pheromones.

Explorer agent algorithm 1. Initialize set t: = 0 For every edge (i, j) set an initial value Tij(t) for trail intensity. Place m ants on the source node. [Generate new explorers at freq. ef] ] 2. Set s: = 1 { tabu list index) for k: = 1 to m do Place starting town of the kth ant in tabuk(s). 3. Repeat until dest’n reached: Set s : = s + 1 for k: =1 to m do Choose the node j to move to with probability pijk (t) Move the kth ant to node j. Update explorer route cost: rk = rk + c(i, j) if (rk > rmax) kill explorerk Insert town j in tabuk(s). At destination go to 4. 4. While s > 1 traverse edge (i, j) T(i, j) = T(i, j) + pe s : = s - 1 5. At source node do: if (pathe = path. Buffer * d) create and send allocator if t > Tmax create and send allocator Evaporation occurs concurrently with exploration

Point-2 -Multipoint Connections • For j destinations, consider as j pt-2 -pt connections with: – same pheromone, i. e. all explorers communicate; – j allocator agents only allocate bandwidth once; – allocator send decision made when % of all j explorer ants agree on spanning tree.

Routing Function Transition probability (mdf): pijk (t) = [Tijk(t) ]-a[C(i, j)]-b / Nk(i, j, t) Nk (i, j, t) = Sj in (S-Tabu(k)) [Tijk(t) ]-a[C(i, j)]-b a, b are control parameters that determine the sensitivity of the algorithm to link cost and pheromone. C(i, j) a function that depends upon the type of traffic, the length and utilization of the link.

Allocator agents • Allocator agents can fail: – bandwidth already allocated by time allocator is sent; – allocator agent backtracks to source rolling back resource allocation and decreases pheromone levels; – decision to re-send allocator made at a later time (a backoff period is observed); – explorer ants continue to search for routes.

Connection monitoring agents • Connection’s quality of service monitored. • Changes in Qo. S cause Connection monitoring agents to be sent into network, laying down q-chemical. D 2 A • E B C

Connection diagnosis agents • Examples of netlets • Sense q-chemical concentrations Mdf D 2 A E B C

Experimental Parameters (fixed) • Number of ants to create = 20 • Frequency of creation = every 10 cycles • Amount of pheromone dropped = 10 • pheromone evaporation rate = 0. 9 • Sample window size = 50 • Emergence criterion = 0. 9

Routing Results • Shortest paths emerged quickly • Mixed pt-2 -pt, pt-2 -mpt routes emerged • Routing responds to changes in environment: – node failure; – link cost changes

Parameter Sensitivity • Bad solutions and stagnation – For high values of a the algorithm enters stagnation behavior very quickly without finding very good solutions. • Bad solutions and no stagnation – a too low, insufficient importance associated with trail. • Good solutions – a , b in the central area (1, 1), (1, 2), (1, 5), (0. 5, 5)

Routing Results • Link cost functions: C(i, j) constant linear threshold quadratic server Five functions studied experimentally: • constant • linear threshold • quadratic • server (1/1 -u) At high occupancy (> 50%) server appeared to give the best results. At low occupancy (<25%) function relatively unimportant.

Diagnosis • Diagnosis agent is ‘hill climbing’ in space of q-chemical. • Very quickly finds high q-chemical concentrations and initiates diagnostic activity. • An example of distributed diagnosis through constructive chemical interference.

Self Adaptation • pheromone and cost sensitivities should vary during search: – avoid premature convergence; – speed up search considerably. • Explorers encode sensitivity values: – fitness of encoding is cost of route; – new agents are created with and use genetically-manipulated values for route finding.

Extensions to current m. Swarm system • Multi-class routing – Higher priority traffic causes pheromone levels of lower priority traffic to decay; – automated re-routing of traffic performed. • Swarms learn to avoid regions of network providing low quality connections.

Other (potential) applications of m-Swarm systems • Behavior-based Network Management; – explorer agents allocate routes; – parent agents monitor “health” of explorers; – congestion agents identify global network congestion; – congestion agents signal re-planning of network; – congestion agent pheromones react with routing agent pheromones in order to cause re-routing after network re-planned.

Futuristic? • Active networks are being researched at: – M. I. T. – U. Penn. – CMU – Georgia Tech. • Management by delegation [Yemini, 91] considered an essential design criterion for next generation network management systems.