Topic 7: Collaborative Distributed Problem Solving application domain type n defining a MAS n cooperation strategies n
The Coordination Problem n Managing the interdependencies between the activities of agents. e. g. ¨ You and I both want to leave the room. ¨ We independently walk towards the door, which can only fit one of us. ¨ I graciously permit you to leave first.
n http: //media. funmansion. com/funmansion/player. p hp? url=http: //media. funmansion. com/content/flv/d 428200 6/crazy_foreign_traffic. flv
1. Understand application domain type (requirements) application characteristics, requirements, constraints ¨ “domain theory” ¨ n n n 2. TOD WOD SOD Task-Oriented Domains Worth-Oriented Domains State-Oriented Domains Define a MAS for the particular system objectives various aspects ¨ design issues / considerations ¨ interaction situations ¨ 3. Define cooperation strategies / examples n AGV case examples 1. simple task allocation, task (re)allocation, jointly lifting, traffic mgt, …
1. Understand application domain type - Domain Theory n Application requirements obviously crucial for defining MAS-based software architecture n Domain theory: study of types / domains of MAS n Task Oriented Domains w à n State Oriented Domains w w à n Agents have tasks to achieve Task (re)distribution Goals specify acceptable final states Side effects Joint plan and schedules Worth Oriented Domains w à Function rating states’ acceptability Joint plan, schedules, and goal relaxation
1. 1 Task Oriented Domains n “Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) n characteristics a set of agents ¨ a set of tasks to achieve ¨ a set of resources ¨ agents can achieve tasks without help or interference from each other ¨ however, agents may benefit by sharing some tasks ¨
Task-oriented Domain: Example Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.
Task-oriented Domain: Example n Imagine you have 3 children, … n Post men Post Office 1 2 a /b /d c / e / f /
Task-oriented Domain: Example n Imagine you have 3 children, … Post men n Database queries n “All female employees making over $50, 000 a year. ” Common Database “All female employees with more than three children. ” 2 1
1. 2 State-Oriented Domain n State-Oriented Domains ¨ n actions lead to changes of system state System objective ==> attain a particular system state either pre-defined/static, either evolving typical additional requirements (e. g. efficiency, …)
State-Oriented Domain e. g. slotted blocks world e. g. furniture moving 1 3 2 1 2 3
1. 3 Worth-Oriented Domains n Worth Oriented Domains ¨ n actions lead to utility: function rating states’ acceptability System objective ==> maximize utility à Joint plan, schedules, and goal relaxation
Worth-Oriented Domains n example: tile world ¨ maximize rewards agents hole B A tile 22 2 5 5 obstacle 2 n example: traffic lights ¨ maximize traffic throughput 34
2. Define a MAS for particular system objectives n identify various ‘elements’ of MAS n design issues / considerations n interaction situations
2. Defining a MAS for particular System objectives n identify various aspects in defining the MAS ¨ starting from system objectives / requirements / constraints ¨ define MAS for reaching the objective(s) n n e reativ c task n n n identify agents identify individual goals identify individual skills (actions, comm, capabilities) identify their relation to resources (insufficient, . . . ) identify their cooperation ¨ n e. g. task allocation, coordination over resources, . . . identify suitable agent architecture
2. Defining a MAS for particular System objectives n identify various aspects in defining the MAS n design issues / considerations n ¨ heterogeneity? n ¨ ¨ ¨ within application boundaries obviously, i. e. if appl. allows all agents can perform tasks, vs. not splittable tasks? specialization? grouping? distributed planning? task distribution predictiveness (anticipatory vs. in situ) n e. g. for avoiding conflicts over resources ¨ e. g. road load, deadlocks, . . .
2. Defining a MAS for particular System objectives n identify various aspects in defining the MAS design issues / considerations n interaction situations n ¨ what is an interaction ? n “dynamic relationship through a set of reciprocal actions” i. e. stem from actions whose consequences have an influence on the future behaviour of the agents n e. g. n ¨ ¨ ¨ agents helping each other data exchange between servers use of a printer by two programs simultaneously
Types of interaction situations n components / criteria for categorization of interaction situations types compatible and incompatible goals ¨ relation to resources ¨ agent capacities / skills ¨
Components of interaction situations ¨ compatible and incompatible goals ¨ ¨ n incompatible antagonist situation ¨ … concurrent goals ? or contradictory or even opposed ? achieving one goal other goal cannot be achieved compatible cooperation situation
Components of interaction situations ¨ … ¨ relation to resources n resources: environmental elements which can be used in carrying out an action ¨ n objects, tools, space, time, … limited resources conflicts ¨ ¨ ¨ agents needing the same tool at the same place in the same time programs sharing a CPU vehicles on busy roads resources needed by A n conflict resolution ¨ ¨ n ¨ resources needed by B many methods § law of the strongest § negotiation e. g. insurances and court houses to resolve conflicts of interest due to accident coordinating actions ¨ resource space devices, regulations, supplementary actions e. g. traffic lights / highway code / … to avoid conflicts
Components of interaction situations ¨ … ¨ capacities of agents in relation to tasks n can an agent carry out a task alone ? or does it need others ? n particular tasks can be carried out either ¨ ¨ by a single agents § e. g. moving a block, doing a calculation, reading a file, … better by several agents / only by several agents § e. g. heavy blocks, different expertise, … beneficial interactions result can be more than sum of parts
Types of interaction situations Resources (each has) Skills Types of situation Involvement Compatible Sufficient Independence Indifference Compatible Sufficient Insufficient Simple collaboration Compatible Insufficient Sufficient Obstruction Cooperative Compatible Insufficient Coordinated collaboration Incompatible Sufficient Pure individual competition Incompatible Sufficient Insufficient Pure collective competition Antagonism Incompatible Insufficient Sufficient Individual conflict over resources Incompatible Insufficient Collective conflict over resources Goals
Types of interaction situations: Examples n Compatible goals, insufficient resources, sufficient skills obstruction ¨ e. g. motorway queue ¨ n Incompatible goals, insufficient resources, sufficient skills individual conflict over resources ¨ e. g. only one liter of water, promotion, . . . ¨ n . . .
3. Cooperation strategies - illustrated in AGV case n application domain type ¨ n task-oriented domain defining a MAS AGV agents, transport agents ¨ individual goals ¨ n n ¨ skills n n ¨ n n … AGV agents: jobs/actions/communication/… transport agents: communication/… resources n ¨ AGV agents: perform tasks transport agents: ensure tasks are performed warehouse (lanes, stock) communication medium cooperation
3. Cooperation strategies - illustrated in AGV case (cont. ) n … n interaction situations (in)compatible goals ¨ (in)sufficient skills ¨ (in)sufficient resources ¨
3. Cooperation strategies - illustrated in AGV case (cont. ) 1. (simple) task allocation CNET ¨ Gradient fields ¨ Dyn. CNET ¨ 2. task (re)distribution ¨ 3. resource allocation - traffic management ¨ 4. negotiation CNET collision avoidance coordination protocol ¨ hull projection ¨ 5. joint action - jointly lift a (heavy) pallet ¨ 6. joint intentions batch order management team formation 2. distributed planning 1.
3. 1 Simple task allocation n situation a task consists of - a pick job (pick up pallet) - a move job (move towards destination) - a drop job (place pallet on target location) a task can be executed by any AGV a task needs to be assigned / allocated to an AGV characteristics “delayed commencement” dynamic communication overhead?
Simple task allocation (cont. ) n situation n collaboration approaches ¨ “Classic approach”: agents coordinate by exchanging messages Contract Net Dyn. CNet ¨ Exploit the agent environment Gradient Fields
Simple task allocation (cont. ) n situation n collaboration approaches 1. Contract Net transport agent issues request for bids (local broadcast) if no response: stronger signal AGV agents can place bids based on current location (distance) other tasks battery level … transport agent assigns task to best bid characteristics …
Simple task allocation (cont. ) n situation n collaboration approaches 1. Contract Net … characteristics communication overhead limited does not take dynamics/opportunities into account no rejection of assigned task no re-allocation if better-suited AGV becomes available issues how should an AGV agent deal with multiple concurrent requests? bid? value of bid? consequences? ok approach for indiviual tasks, what for large numbers of tasks? one task at a time
Simple task allocation (cont. ) n situation n collaboration approaches 2. Dynamic Contract Net (Dyn. CNET) contract net, but taking into account dynamics characteristics communication overhead can take dynamics/opportunities into account rejection of assigned task re-allocation if better-suited AGV becomes available issues how should an AGV agent deal with multiple concurrent requests? bid? value of bid? consequences? ok approach for indiviual tasks, what for large numbers of tasks? one task at a time
New Task Becomes Available
New AGV Becomes Available
Dyn. CNET = Dynamic Contract Net
Simple task allocation (cont. ) n situation n collaboration approaches 3. Gradient Fields coordination through the environment physical environment § Restricts how agents can exploit the environment ¨ agent environment § virtual environment layer § enables agents to share information and coordinate their behavior ¨ characteristics cfr. Dyn. CNET reduced complexity of AGV agent!
Agent Environment for Agents
Exploit the Agent Environment Task Assignment (Field-Based Approach)
Exploit the Agent Environment Task Assignment
Task Allocation experiments Test Setting AGV System n AGV system “fish handling” (real scenario) 134 x 134 m; 56 pick/drop locations ¨ 14 AGVs; 0. 7 m/s; load manipulation 5 s ¨ 140 transports/hour (standard test profile) ¨ n Tests Communication load ¨ Average waiting time ¨ Stress test: perform fixed number of transports as quick as possible (e. g. handle the arrival of a truck with loads) ¨
Test Results Communication Load Average Waiting Time
Finished Transports in Stress Test
Two Qualities: Flexibility and Robustness n Flexibility (adapt to changes in the environment) Fi. TA: implicitly represented in the fields ¨ Dyn. CNET: explicit points of choice ¨ n Robustness (to message loss) Fi. TA: graceful degradation ¨ Dyn. CNET: requires additional functionality ¨
3. 2 Task (re)distribution n situation AGV agents have several tasks assigned to them a set of AGV agents could redistribute tasks amongst each other for better results e. g. after using several CNET protocols, the following assignment exists: AGV 1: task 1 (NE corner), task 2 (SW corner) AGV 2: task 3 (NE corner), task 4 (SW corner) redistributing the tasks could seriously enhance task execution time AGV 1: task 1 (NE corner), task 3 (NE corner) AGV 2: task 2 (SW corner), task 4 (SW corner)
Task (re)distribution (cont. ) n situation n typical situation for negotiation in Task-Oriented Domains …
Task (re)distribution (cont. ) Task-Oriented Domains (TODs) Defined n a TOD is a triple where <T, Ag, c> T is the (finite) set of all possible tasks ¨ Ag = {1, …, n} is the set of participating agents ¨ c = Ã(T) ú + defines the cost of executing each subset of tasks ¨ n an encounter is a collection of tasks <T 1, …, Tn> where Ti Í T for each i Î Ag
Deals in TODs n n n given encounter <T 1, T 2>, a deal is an allocation of the tasks T 1 È T 2 to the agents 1 and 2 the cost to i of deal d = <D 1, D 2> is c(Di), and will be denoted costi(d) the utility of deal d to agent i is: utilityi(d) = c(Ti ) – costi(d) the conflict deal, Q, is the deal <T 1, T 2> consisting of the tasks originally allocated. note that utilityi(Q) = 0 for all i Î Ag deal d is individual rational if it weakly dominates the conflict deal
Negotiation n ”The process of several agents searching for an agreement” n Reaching consensus ”Rules of Encouter” by Rosenchein and Zlotskin, 1994
Complexity of Negotiations n Some attributes that make the negotiation process complex are: n Multiple attributes: ¨ ¨ n Single attribute (price) – symmetric scenario. Multiple attributes – several inter-related attributes, e. g. buying a car. The number of agents and the way they interact: ¨ ¨ ¨ One-to-one, e. g. single buyer and single seller. Many-to-one, e. g. multiple buyers and a single seller, auctions. Many-to-many, e. g. multiple buyers and multiple sellers.
Negotiation Components n Any negotiation setting will have 4 components: ¨ Negotiation set: represents the space of possible proposals that agents can make ¨ Protocol: defines the legal proposals that agents can make ¨ Collection of strategies: (one for each agent) determines what proposals the agent will make ¨ Rule: to determine when an agreement has been reached
The Negotiation Set n the set of deals over which agents negotiate are those that are: individual rational ¨ pareto efficient ¨
The Negotiation Set Illustrated
Mechanisms, Protocols, Strategies n Negotiation is governed by a mechanism or a protocol: ¨ defines the ”rules of encounter” between the agents ¨ the public rules by which the agents will come to agreements. n Given a particular protocol, how can a particular strategy be designed that individual agents can use?
Monotonic Concession Protocol n Negotiation proceeds in rounds n On round 1, agents simultaneously propose a deal from the negotiation set. n Agreement is reached if one agent finds that the deal proposed by the other agent is at least as good or better than its proposal. Ai best deal Aj best deal
Monotonic Concession Protocol n If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals. n In round u+1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal proposed at time u. n If neither agent concedes, then negotiation terminates with a conflict deal.
Monotonic Concession Protocol n Advantages: Symmetrically distributed (no agent plays a special role) ¨ Ensures convergence ¨ It will not go on indefinitely ¨ n Disadvantages: Agents can run into conflicts ¨ Inefficient – no quarantee that an agreement will be reached quickly ¨
Key Questions n 3 key questions to be answered: ¨ What should an agent’s first proposal be? It’s most preferred deal. ¨ On any given round, who should concede? The agent least willing to risk conflict. ¨ If an agent concedes, then how much should it concede? Just enough to change the balance of risk.
The Risk Factor n One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point. How much am I willing to risk a conflict? Maximum loss from conflict Maximum loss from concession Conflict deal Ai best deal Aj best deal
The Zeuthen Strategy n Uses the risk evaluation strategy n Suppose you have conceded a lot. Then: Your proposal is now close to conflict deal. ¨ You are more willing to risk conflict. ¨ n An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low. n Degree of willingness to risk a conflict can be defined as: Riskti = utility i loses by conceding and accepting j’s offer utility i loses by not conceding and causing conflict
Nash Equilibrium… n the Zeuthen strategy is in Nash equilibrium ¨ n under the assumption that one agent is using the strategy the other can do no better than use it himself… interesting to the designer of automated agents ¨ publicly known strategy
About MCP and Zeuthen Strategies n Advantages: Simple and reflects the way human negotiations work. ¨ Stability – in Nash equilibrium – if one agent is using the strategy, then the other can do no better than using it him/herself. ¨ n Disadvantages: Computationally expensive – players need to compute the entire negotiation set. ¨ Communication burden – negotiation process may involve several steps. ¨
3. 3 Resource allocation - traffic management (cont. ) n situation n approach negotiation protocols delegate MAS - see later
3. 4 Collision avoidance n situation AGV autonomously follow their own routes towards source / target locations warehouse layout may contain locations where only 1 AGVs can pass at a time e. g. cross roads, close lanes
Collision avoidance (cont. ) n situation n approaches ¨ collision avoidance communication protocols n many possibilities ¨ ¨ n local decision e. g. based on priority rules (priority to the right) CNET for bidding on ‘critical section’ quite complex in non-trivial situations ¨ ¨ ¨ n vehicles complex warehouse layouts dynamic set of AGVs to take into account needs to be integrated seamlessly in agent architecture… deadlock?
Collision avoidance (cont. ) n situation n approaches ¨ collision avoidance communication protocols ¨ hull projection n n environment as controlling entity environment mediates between AGV requests to use a lane more easily extensible for more vehicles / complex layouts more easily integrated in agent architecture
Hull projection through the virtual environment
3. 5 Joint action - jointly lift a (heavy) pallet n situation heavy pallet needs to be transported ¨ heavy = requires two AGVs ¨ AGVs need to agree on this ¨ AGVs need to stick to the agreement ¨ n an AGV cannot suddenly drop the pallet because it sees another opportunity (e. g. another job)
Joint action - jointly lift a (heavy) pallet (cont. ) n situation n approach ¨ joint intentions (Jennings, Tambe) n Recall ¨ n intentions important for individual practical reasoning Now: ¨ ¨ ¨ intentions also important for coordination distinguish § individual intentions (that may be coordinated) § from intentions to cooperatively and coordinatedly achieve a goal as a team commitment associated with an intention § future directed, persitent, should not be dropped for no reason § conventions exist that regulate when it‘s appropriate to drop an intention § e. g. when lifting a heavy pallet together
Coordination through joint intentions (cont. ) n Agents need n n ¨ individual commitments, and joint commitment to overall goal State of joint commitment is distributed over agents n Convention regulates e. g. when joint commitment may be dropped and how other agents need to be informed about change of mind. n More formally: Joint persistant goals (JPGs) JPG = (goal φ, motivation for goal ψ) e. g. : φ = „having heavy object lifted onto truck“; ψ = „later transportation“
Conventions next to intentions n Initially each agent believes φ has not been satsified, and ¨ believes possible. To. Do φ ¨ n Until termination condition is met, each agent has goal φ (more precisely intention φ) n Termination condition: It is mutually believed that either φ is satisfied ¨ φ is impossible ¨ ψ is no longer valid ¨ n …
n … n Until termination condition is met: ¨ if an agent believes that either n n n ¨ the goal is met or the goal is impossible or the motivation no longer holds it has the goal to make this mutually believed (the goal to convince all others about that)
n Examples for JPG-like architectures: ARCHON (Jennings e. a. ) ¨ Steam (Tambe) ¨ n Both encode conventions formally as rules, thus integrating them into the „normal“ reasoning process
3. 6 Batch order handling n situation peek load in warehouse ¨ e. g. several loaded trucks need to be cleared 10 s of pallets need to be transported some pallets should stay together (e. g. same colour material) ¨ ¨ complex tasks may need to be decomposed? n stack of pallets --> smaller stacks or individual pallets ¨ narrow unload location - coordination required ¨ humans would make a team for efficiency? n unloading chain?
Batch order handling (cont. ) n n situation approach organisation-based approach ¨ coordinated planning ¨
Batch order handling (cont. ) n n head (unload) situation approach ¨ body (pass on) organisation-based approach n inspired by human organisations define roles, assign/negotiate about roles commitment n body (pass on) e. g. action chain n n ¨ tail (drop off) roles: head of chain tail of chain body of chain transport
Batch order handling (cont. ) n n situation approach ¨ organisation-based approach ¨ coordinated planning n centralized (locally or globally) ¨ ¨ n plan building and distribution plan merging distributed ¨ ¨ (G)PGP - (Generalized) Partial Global Planning (Durfee e. a. ) Shared. Plans (Grosz, Kraus)
Coordinated planning ¨ Centralized Planning n n plan building and distribution plan merging ¨ ¨ ¨ Distributed Planning n n n each agent makes local, partial plan (PP) the PPs are communicated to central coordinator it resolves the conflicts and generates a conflict-free global plan (GP). GP is communicated to all agents. e. g. Georgeff in AAAI conference 1983 (National Conf on AI) no central controller each agent has some knowledge or model of other agents It has the capability to piece the PPs together into an agreeable GP e. g. Victor Lesser and group (University of Massachusettes, 1981) Critique: agents share and process a huge amount of information ¨ requires lots of computing and communication resources ¨ centralised approaches introduce a single point of failure ¨ sometimes plans can also become obsolete very quickly ¨
Conclusion n collaborative distributed problem solving ¨ at the heart of multi-agent systems n just a small sample of possible situations / cooperation strategies here … n distributed problem solving solves problems but creates new challenges ¨ many coordination situations ¨ many possible solutions ¨ n in the end: integration in one agent architecture ¨ coordination and coherence !! ¨ n n coordination Managing inter-dependencies between the activities of agents coherence: “how well the MAS behaves as a unit along some dimension of evaluation” (Bond et al. )
Скачать презентацию Topic 7 Collaborative Distributed Problem Solving application domain
b67c5d64623f6a6941820ffca4e84227.ppt