SAC 2002 Tutorial Henry Hexmoor Svet Brainov University of Arkansas Engineering Hall, Room 328 Fayetteville, AR 72701 University at Buffalo 210 Bell Hall Buffalo, NY 14260
Hexmoor&Braynov Contents Multiagents: Formal and Economic Morning: Basics n I. Introduction: from DAI to Multiagecy 1. History and perspectives on multiagents (Henry) 2. Architectural theories (Henry) 3. Agent Oriented Software Engineering (Henry) 4. Mobility, reliability, and fault-tolerance (Henry) n II. Enabling Technologies 5. Game Theoretic and Decision Theoretic Agents(Svet) 6. Communication, security (Svet) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov n Multiagents: Formal and Economic Contents: Continued Afternoon: Issues III. Enabling Technologies 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles (Henry) 8. Benevolence, Preference, Power, Trust (Svet) 9. Communication, Security(Svet) 10. Agent Adaptation and Learning (Svet) n IV. Closing 11. Trends and open questions (Svet) 12. Concluding Remarks (Svet) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Definitions 1. An agent is an entity whose state is viewed as consisting of mental components such as beliefs, capabilities, choices, and commitments. [Yoav Shoham, 1993] 2. An entity is a software agent if and only if it communicates correctly in an agent communication language. [Genesereth and Ketchpel, 1994] 3. Intelligent agents continuously perform three functions: perception of dynamic conditions in the environment; action to affect conditions in the environment; and reasoning to interpret perceptions, solve problems, draw SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Definitions 5. An agent is anything that can be viewed as (a)Perceiving its environment, and (b) Acting upon that environment [Russell and Norvig, 1995] 6. A computer system that is situated in some environment and is capable of autonomous action in its environment to meet its design objectives. [Wooldridge, 1999] SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agents: A working definition An agent is a computational system that interacts with one or more counterparts or real-world systems with the following key features to varying degrees: • Autonomy • Reactiveness • Pro-activeness • Social abilities e. g. , autonomous robots, human assistants, service agents The need is for automation and distributed use of online resources SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Test of Agenthood [Huhns and Singh, 1998] “A system](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-8.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Test of Agenthood [Huhns and Singh, 1998] “A system")
Hexmoor&Braynov Multiagents: Formal and Economic Test of Agenthood [Huhns and Singh, 1998] “A system of distinguished agents should substantially change semantically if a distinguished agent is added. ” SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Agents vs. Objects “Objects with attitude” [Bradshaw, 1997] n](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-9.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Agents vs. Objects “Objects with attitude” [Bradshaw, 1997] n")
Hexmoor&Braynov Multiagents: Formal and Economic Agents vs. Objects “Objects with attitude” [Bradshaw, 1997] n Agents are similar to objects since they are computational units that encapsulate a state and communicate via message passing n Agents differ from objects since they have a strong sense of autonomy and are active versus passive. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agent Oriented Programming, Yoav Shoham AOP principles: 1. The state of an object in OO programming has no generic structure. The state of an agent has a “mentalistic” structure: it consists of mental components such as beliefs and commitments. 2. Messages in object-oriented programming are coded in an application-specific ad-hoc manner. A message in AOP is coded as a “speech act” according to a standard agent communication language that is application-independent. SAC 2002 Tutorial 3/13/02
Multiagents: Formal and Economic Agent Oriented Programming Extends Peter Chen’s ER model, Gerd Wagner Hexmoor&Braynov • Different entities may belong to different epistemic categories. There agents, events, actions, commitments, claims, and objects. • We distinguish between physical and communicative actions/events. Actions create events, but not all events are created by actions. • Some of these modeling concepts are indexical, that is, they depend on the perspective chosen: in the perspective of a particular agent, actions of other agents are viewed as events, and commitments of other agents are viewed as claims against them. SAC 2002 Tutorial 3/13/02
Multiagents: Formal and Economic Agent Oriented Programming Extends Peter Chen’s ER model, Gerd Wagner Hexmoor&Braynov • • • In the internal perspective of an agent, a commitment refers to a specific action to be performed in due time, while a claim refers to a specific event that is created by an action of another agent, and has to occur in due time. Communication is viewed as asynchronous point-to-point message passing. We take the expressions receiving a message and sending a message as synonyms of perceiving a communication event and performing a communication act. There are six designated relationships in which specifically agents, but not objects, participate: only an agent perceives environment events, receives and sends messages, does physical actions, has Commitment to perform some action in due time, and has Claim that some action event will happen in due time. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agent Oriented Programming Extends Peter Chen’s ER model, Gerd Wagner • • An institutional agent consists of a certain number of (institutional, artificial and human) internal agents acting on behalf of it. An institutional agent can only perceive and act through its internal agents. Within an institutional agent, each internal agent has certain rights and duties. There are three kinds of duties: an internal agent may have the duty to full commitments of a certain type, the duty to monitor claims of a certain type, or the duty to react to events of a certain type on behalf of the organization. A right refers to an action type such that the internal agent is permitted to perform actions of that type on behalf of the organization. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Agent Typology Multiagents: Formal and Economic • Human agents: Person, Employee, Student, Nurse, or Patient • Artificial agents: owned and run by a legal entity • Institutional agents: a bank or a hospital • Software agents: Agents designed with software • Information agent: Data bases and the internet • Autonomous agents: Non-trivial independence • Interactive/Interface agents: Designed for interaction • Adaptive agents: Non-trivial ability for change • Mobile agents: code and logic mobility SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Agent Typology Multiagents: Formal and Economic • Collaborative/Coordinative agents: Non-trivial ability for coordination, autonomy, and sociability • Reactive agents: No internal state and shallow reasoning • Hybrid agents: a combination of deliberative and reactive components • Heterogenous agents: A system with various agent sub-components • Intelligent/smart agents: Reasoning and intentional notions • Wrapper agents: Facility for interaction with nonagents SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multi-agency A multi-agent system is a system that is made up of multiple agents with the following key features among agents to varying degrees of commonality and adaptation: • Social rationality • Normative patterns • System of Values e. g. , HVAC, e. Commerce, space missions, Soccer, Intelligent Home, “talk” monitor The motivation is coherence and distribution of resources. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Applications of Multiagent Systems n Electronic commerce: B 2 B, Info. Flow, e. CRM n Network and system management agents: E. g. , The telecommunications companies n Real-time monitoring and control of networks: ATM n Modeling and control of transportation systems: Delivery n Information retrieval: online search n Automatic meeting scheduling n Electronic entertainment: e. Dog SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Applications of Multiagent Systems (cont. ) n Decision and logistic support agents: Military and Utility Companies n Interest matching agents: Commercial sites like Amazon. com n User assistance agents: E. g. , MS office assistant n Organizational structure agents: Supply-chain ops n Industrial manufacturing and production: manufacturing cells n Personal agents: emails n Investigation of complex social phenomena such as evolution of roles, norms, and organizational structures SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Summary of Business Benefits • Modeling existing organizations and dynamics • Modeling and Engineering E-societies • New tools for distributed knowledge-ware SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Three views of Multi-agency Constructivist: Agents are rational in the sense of Newell’s principle of individual rationality. They only perform goals which bring them a positive net benefit without regard to other agents. These are selfinterested agents. Sociality: Agents are rational in the Jennings’ principle of social rationality. They perform actions whose joint benefit is greater than its joint loss. These are self-less, responsible agents. Reductionist: Agents which accept all goals they are capable of performing. These are benevolent agents. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multi-agency: allied fields DAI MAS: (1) online social laws, (2) agents may adopt goals and adapt beyond any problem DPS: offline social laws CPS: (1) agents are a ‘team’, (2) agents ‘know’ the shared goal 1. In DAI, a problem is being automatically decomposed among distributed nodes, whereas in multi-agents, each agent chooses to whether to participate. 2. Distributed planning is distributed and decentralized action selection whereas in multi-agents, agents keep their own copies a plan that might include others. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multi-agent assumptions and goals • Agents have their own intentions and the system has distributed intentionality • Agents model other agents mental states in their own decision making • Agent internals are of less central than agents interactions • Agents deliberate over their interactions • Emergence at the agent level and at the interaction level are desirable • The goals is to find some principles-for or principled ways to explore interactions SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Origins of Multi-agent systems • Carl Hewitt’s Actor model, 1970 • Blackboard Systems: Hearsay (1975), BB 1, GBB • Distributed Vehicle Monitoring System (DVMT, 1983) • Distributed AI • Distributed OS SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic MAS Orientations Sociology Computational Organization Theory Databases Formal AI Economics Distributed Problem Solving Psychology Systems Theory SAC 2002 Tutorial Cognitive Science Distributed Computing 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Conferences • ICMAS 96, 98, 00, 02 • Autonomous Agents 96, 97, 98, 99, 00, 02 • CIA, ATAL, CEEMAS SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multi-agents in the large versus in the small • In the small: (Distributed AI) A handful of “smart” agents with emergence in the agents • In the large: 100+ “simple” agents with emergence in the group: Swarms (Bugs) http: //www. swarm. org/ SAC 2002 Tutorial 3/13/02
Autonomy Henry Hexmoor’s Tree of Research Issues Purposefulness Learning Action Selection Timeliness Habituation Commitment Skill formation Values, Norms, Obligations itec Arc h Teamwork Cooperation Social attitudes ture Automaticity Agents Perception Inference
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Abstract Architecture states actions Environment SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Architectures • Deduction/logic-based • Reactive • BDI • Layered (hybrid) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Abstract Architectures n An abstract model: <States, Action, S* A> n An abstract view u. S u. A n = {s 1, s 2, …} – environment states = {a 1, a 2, …} – set of possible actions This allows us to view an agent as a function action : S* A SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Logic-Based Architectures These agents have internal state n See and next functions and model decision making by a set of deduction rules for inference n see : S P next : D x P D action : D A n n Use logical deduction to try to prove the next action to take Advantages u. Simple, n elegant, logical semantics Disadvatages u. Computational complexity u. Representing the real world SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Reactive Architectures n Reactive Architectures do not use u symbolic world model u symbolic reasoning n An example is Rod Brooks’s subsumption architecture n Advantages u Simplicity, n computationally tractable, robust, elegance Disadvantages u Modeling SAC 2002 Tutorial limitations, correctness, realism 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Reflexive Architectures: simplest type of reactive architecture n Reflexive agents decide what to do without regard to history – purely reflexive action : P A n Example - thermostat action(s) = SAC 2002 Tutorial { off on if temp = OK otherwise 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Reflex agent without state (Russell and Norvig, 1995) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Reflex agent with state (Russell and Norvig, 1995) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Goal-oriented agent: a more complex reactive agent (Russell and Norvig, 1995) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Utility-based agent: a complex reactive agent (Russell and Norvig, 1995) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic BDI: a Formal Method • Belief: states, facts, knowledge, data • Desire: wish, goal, motivation (these might conflict) • Intention: a) select actions, b) performs actions, c) explain choices of action (no conflicts) • Commitment: persistence of intentions and trials • Know-how: having the procedural knowledge for carrying out a task SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Belief-Desire-Intention Environment belief revision sense act Beliefs generate options Desires SAC 2002 Tutorial filter Intentions 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Why is BDI a Formal Method? • BDI is typically specified in the language of modal logic with possible world semantics. • Possible worlds capture the various ways the world might develop. Since the formalism in [Wooldridge 2000] assumes at least a KD axiomatization for each of B, D, and I, each of the sets of possible worlds representing B, D and I must be consistent. • A KD 45 logic with the following axioms: • K: BDI(a, f j, t) (BDI(a, f, t) BDI(a, j, t)) • D: BDI(a, f, t) not BDI(a, not f, t) • 4: B(a, f, t) B( B(a, f, t) ) • 5: (not B(a, f, t)) B( not B(a, f, t)) • K&D is the normal modal system SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A simplified BDI agent algorithm 1. B = B 0; 2. I : = I 0; 3. while true do 4. get next percept r; 5. B : = brf(B, r); // belief revision 6. D: =options(B, D, I, O); // determination of desires 7. I : = filter(B, D, I, O); // determination of intentions 8. : = plan(B, I); // plan generation 9. execute 10. end while SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Correspondences Multiagents: Formal and Economic • Belief-Goal compatibility: Des Bel • Goal-Intention Compatibility: Int Des • Volitional Commitment: Int Do • Awareness of Goals and Intentions: Des Bel. Des Int Bel. Int SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Layered Architectures n Layering is based on division of behaviors into automatic and controlled. n Layering might be Horizontal (I. e. , I/O at each layer) or Vertical (I. e. , I/O is dealt with by single layer) n Advantages are that these are popular and fairly intuitive modeling of behavior n Dis-advantages are that these are too complex and nonuniform representations SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agent-Oriented Software Engineering u AOSE is an approach to developing software using agent-oriented abstractions that models high level interactions and relationships. u Agents are used to model run-time decisions about the nature and scope of interactions that are not known ahead of time. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Designing Agents: Multiagents: Formal and Economic Recommendations from H. Van Dyke Parunak’s (1996) “Go to the Ant”: Engineering Principles from Natural Multi-Agent Systems, Annals of Operations Research, special issue on AI and Management Science. 1. Agents should correspond to things in the problem domain rather than to abstract functions. 2. Agents should be small in mass (a small fraction of the total system), time (able to forget), scope (avoiding global knowledge and action). 3. The agent community should be decentralized, without a single point of control or failure. 4. Agents should be neither homogeneous nor incompatible, but diverse. Randomness and repulsion are important tools for establishing and maintaining this diversity. 5. Agent communities should include a dissipative mechanism to whose flow they can orient themselves, thus leaking entropy away from the macro level at which they do useful work. 6. Agents should have ways of caching and sharing what they learn about their environment, whether at the level of the individual, the generational chain, or the overall community organization. 7. Agents should plan and execute concurrently rather than sequentially. SAC 2002 Tutorial 3/13/02
Organizations Human organizations are several agents, engaged in multiple goal-directed tasks, with distinct knowledge, culture, memories, history, and capabilities, and separate legal standing from that of individual agents Computational Organization Theory (COT) models information production and manipulation in organizations of human and computational agents
Hexmoor&Braynov Multiagents: Formal and Economic Management of Organizational Structure n Organizational constructs are modeled as entities in multiagent systems n Multiagent systems have built in mechanisms for flexibly forming, maintaining, and abandoning organizations n Multiagent systems can provide a variety of stable intermediary forms in rapid systems development SAC 2002 Tutorial 3/13/02
7. 2. 1 Agent and Agency
Hexmoor&Braynov Multiagents: Formal and Economic AOSE Considerations n What, how many, structure of agent? n Model of the environment? n Communication? Protocols? Relationships? Coordination? SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Stages of Agent-Oriented Software Engineering A. Requirements: provided by user B. Analysis: objectives and invariants C. Design: Agents and Interactions D. Implementation: Tools and techniques SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Ko. AS- Bradshaw, et al ØKnowledge (Facts) represent Beliefs in which the agent has confidence about ØFacts and Beliefs may be held privately or be shared. ØDesires represent goals and preferences that motivate the agent to act ØIntentions represent a commitment to perform an action. ØThere is no exact description of capabilities ØLife cycle: birth, life, and death (also a Cryogenic state) ØAgent Types: Ka. OS, Mediation (Ka. OS and outside) , Proxy (mediator between two KAOS agents), Domain Manager (agent registration), and Matchmaker (mediator of services) ØOmitted: Emotions, Learning, agent relationships, 3/13/02 SAC 2002 Tutorial
Hexmoor&Braynov Multiagents: Formal and Economic Gaia- Wooldridge, et al The Analysis phase: • Roles model: - Permissions (resources) - Responsibilities (Safety properties and Liveliness properties) - Protocols • Interactions model: purpose, initiator, responder, inputs, outputs, and processing of the conversation The Design phase: Agent model • Services model Acquaintance model Omitted: Trust, Fraud, Commitment, and Security. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic TAEMS: Keith Decker and Victor Lesser n The agents are simple processors. n Internal structure of agents include (a) beliefs (knowledge) about task structure, (b) states, (c) actions, (d) a strategy which is constantly being updated, of what methods the agent intends to execute at what time. n Omitted: Roles, Skills or Resources. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic BDI based Agent-Oriented Methodology (KGR) Kinny, Georgeff and Rao n External viewpoint: the social system structure and dynamics. Agent Model + Interaction Model. § Independent of agent cognitive model and communication § n Internal viewpoint: the Belief Model, the Goal Model, and the Plan Model. Beliefs: the environment, internal state, the actions repertoire § Goals: possible goals, desired events § Plans: state charts § SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Ma. SE – Multi-agent Systems Engineering, De. Loach Domain Level Design (Use Ag. ML for Agent type Diagram, Communication Hierarchy Diagram, and Communication class Diagrams. ) n Agent Level Design (Use Ag. DL for agent conversation) n Component Design Ag. DL n System Design Ag. ML n Languages: Ag. ML (Agent Modeling Language- a graphical language) Ag. DL (Agent Definition Language- the system level behavior and the internal behavior of the agent) n Rich in communication, poor in social structures n SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Scott De. Loach’s Ma. SE Roles Tasks Agent Class Diagram Sequence Diagrams Conversation Diagram Internal Agent Diagram Deployment Diagram SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic The TOVE Project (1998) ; Mark Fox, et al. • Organizational hierarchy: Divisions and sub-divisions • Goals, sub-goals, their hierarchy (using AND & OR) • Roles, their relations to skills, goals, authority, processes, policies • Skills, and their link to roles • Agents, their affiliation with teams and divisions Commitment, Empowerment • Communication links between agents: sending and receiving information. Communication at three levels: information, intentions (ask, tell, deny…), and conventions (semantics). Levels 2 & 3 are designed using speech act. • Teams as temporary group of agents • Activities and their states, the connection to resources and the constraints. • Resources and their relation to activities and activities states • Constraints on activities (what activities can occur at a specific situation and a specific time) • Time and the duration of activities. Actions occur at a point in time and they have duration. • Situation Shortcomings: central decision making SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agent-Oriented Programming (AOP): Yoav Shoham 1. AGENT 0 is the first AOP and the logical component of this language is a quantified multi-modal logic. 2. Mental state: beliefs, capabilities, and commitments (or obligations). 3. Communication: ‘request’ (to perform an action), ‘unrequest’ (to refrain from action), and ‘inform’ (to pass information). SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic The MADKIT Agent Platform Architecture: Olivier Gutknecht Jacques Ferber n Three core concepts : agent, group, and role. n Interaction language n Organizations: a set of groups SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agents [Singh, 1999] A computation that can change](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-63.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agents [Singh, 1999] A computation that can change")
Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agents [Singh, 1999] A computation that can change its location of execution (given a suitable underlying execution environment), both ucode uprogram state [Papaioannou, 1999] A software agent that is able to migrate from one host to another in a computer network is a mobile agent. [IBM] Mobile network agents are programs that can be dispatched from one computer and transported to a remote computer for execution. Arriving at the remote computer, they present their credentials and obtain access to local services and data. The remote computer may also serve as a broker by bringing together agents with similar interests and compatible goals, thus providing a meeting place at which agents can interact. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agent Origins - Batch Jobs - Distributed Operating System (migration is transparent to the user. ) - Telescript [General Magic, Inc. USA, 1994] migration of an executing program for use of local resources SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A paradigm shift: Distributed Systems versus mobile code Instead of masking the physical location of a component, mobile code infrastructures make it evident. Code mobility is geared for Internet-scale systems. . . unreliable Programming is location aware. . . location is available to the programmer Mobility is a choice. . . migration is controlled by the programmer or at runtime by the agent Load balancing is not the driving force. . . instead flexibility, autonomy and disconnected operations are key factors SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A paradigm comparison: 2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task Remote Computation In remote computation, components in the system are static, whereas logic can be mobile. For example, component A, at Host HA, contains the required logic L to perform a particular task T, but does not have access to the required resources R to complete the task. R can be found at HB, so A forwards the logic to component B, which also resides at HB. B then executes the logic before returning the result to A. E. g. , batch entries. HA L, T HA L SAC 2002 Tutorial HB R L result HB Compute R 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A paradigm comparison: 2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task Code on Demand In Code on Demand, component A already has access to resource R. However, A (or any other components at Host A) has no idea of the logic required to perform task T. Thus, A sends a request to B for it to forward the logic L. Upon receipt, A is then able to perform T. An example of this abstraction is a Java applet, in which a piece of code is downloaded from a web server by a web browser and then executed. HA R Compute SAC 2002 Tutorial HA R HB L Send L L HB L 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A paradigm comparison: 2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task Mobile Agents With the mobile agent paradigm, component A already has the logic L required to perform task T, but again does not have access to resource R. This resource can be found at HB. This time however, instead of forwarding/requesting L to/from another component, component A itself is able to migrate to the new host and interact locally with R to perform T. This method is quite different to the previous two examples, in this instance an entire component is migrating, along with its associated data and logic. This is potentially the most interesting example of all the mobile code abstractions. There are currently no contemporary examples of this approach, but we examine its capabilities in the next section. HA L SAC 2002 Tutorial HB R A moves A returns HB R Compute 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic A paradigm comparison: 2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task Client/Server is a well known architectural abstraction that has been employed since the first computers began to communicate. In this example, B has the logic L to carry out Task T, and has access to resource R. Component A has none of these, and is unable to transport itself. Therefore, for A to obtain the result of T, it must resort to sending a request to B, prompting B to carry out Task T. The result is then communicated back to A when completed. HA HA HB L, R request result SAC 2002 Tutorial HB L, R Compute 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Problems in distributed Systems: J. Waldo Latency: Most obvious, Least worrisome Memory: Access, Unable to use pointers, Because memory is both local and remote, call types have to differ, No possibility of shared memory Partial Failure: Is a defining problem of distributed computing, Not possible in local computing, Concurrency: Adds significant overhead to programming model, No programmer control of method invocation order we should treat local and remote objects differently. Waldo, J. , Wyant, G. , Wollrath, A. , Kendall, S. , “A note on distributed computing”, Sun Microsystems Technical Report SML 94 -29, 1994. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agent Toolkit from IBM: Basic concepts n. Aglet. An aglet is a mobile Java object that visits aglet-enabled hosts in a computer network. It is autonomous, since it runs in its own thread of execution after arriving at a host, and reactive, because of its ability to respond to incoming messages. n. Proxy. A proxy is a representative of an aglet. It serves as a shield for the aglet that protects the aglet from direct access to its public methods. The proxy also provides location transparency for the aglet; that is, it can hide the aglet’s real location of the aglet. n. Context. A context is an aglet's workplace. It is a stationary object that provides a means for maintaining and managing running aglets in a uniform execution environment where the host system is secured against malicious aglets. One node in a computer network may run multiple servers and each server may host multiple contexts. Contexts are named and can thus be located by the combination of their server's address and their name. n. Message. A message is an object exchanged between aglets. It allows for synchronous as well as asynchronous message passing between aglets. Message passing can be used by aglets to collaborate and exchange information in a loosely coupled fashion. n. Future reply. A future reply is used in asynchronous message-sending as a handler to receive a result later asynchronously. n. Identifier. An identifier is bound to each aglet. This identifier is globally unique and immutable throughout the lifetime of the aglet. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Mobile Agent Toolkit from IBM: Basic operations n. Creation. The creation of an aglet takes place in a context. The new aglet is assigned an identifier, inserted into the context, and initialized. The aglet starts executing as soon as it has been successfully initialized. n. Cloning. The cloning of an aglet produces an almost identical copy of the original aglet in the same context. The only differences are the assigned identifier and the fact that execution restarts in the new aglet. Note that execution threads are not cloned. n. Dispatching an aglet from one context to another will remove it from its current context and insert it into the destination context, where it will restart execution (execution threads do not migrate). We say that the aglet has been “pushed” to its new context. n. Retraction. The retraction of an aglet will pull (remove) it from its current context and insert it into the context from which the retraction was requested. n. Activation and deactivation. The deactivation of an aglet is the ability to temporarily halt its execution and store its state in secondary storage. Activation of an aglet will restore it in a context. n. Disposal. The disposal of an aglet will halt its current execution and remove it from its current context. n. Messaging between aglets involves sending, receiving, and handling messages synchronously as well as asynchronously. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Autonomy • Target and Context: Autonomy is only meaningful in terms of specific targets and within given contexts. • Capability: Autonomy only makes sense if an agent has a capability toward a target. E. g, a rock is not autonomous • Sources of Autonomy: Endogenous: Self liberty, Desire, Experience, Motivations Exogenous: Social, Deontic liberty, Environments • Implementations: Off-line and by design, Online with fixed cost analysis, Online learning
Perspectives on Autonomy Cognitive Science and AI Organizational Science Communication Software Engineering
Hexmoor&Braynov Multiagents: Formal and Economic Autonomy and Communication n. Detection and expression of autonomies requires sharing understanding of social roles and personal relationships among the participating agents, e. g. , agents with positive relationships will would change their autonomies to accommodate one another n. The form of the directive holds clues for autonomy, e. g. , specificity in “Do x with a wrench and slowly. ” n. The content of the directive and the responses to it contribute to the autonomy, e. g. , “Do x soon. ” n. An agent’s internal mechanism for autonomy determination affects the detection, expression, and harmony of autonomies, e. g. , an agent’s moods, drives, temperaments, … SAC 2002 Tutorial 3/13/02
Situated Autonomy and Action Selection enablers sensory data communications beliefs situated autonomy physical goal physical act intention communication goal communication intention
Shared Autonomy between an Air Traffic Control assistant agent and the human operator- 1999
Autonomy Computation Collision: Autonomy = (Collision. Priority / 4. 0) + (((|Collision. Priority – 4. 0|) * t) / T) Landing: If 3. 0 <= Landing. Priority <= 4. 0: Autonomy = 1. 0 If Landing. Priority < 3. 0: Autonomy = (Landing. Priority/4. 0) + (((|Landing. Priority – 4. 0|) * t) / 2)
Team- Building Intuition • Drivers on the road are generally not a team • Race driving in a “draft” is a team • 11 soccer players declaring to be a team are a team • Herding sheep is generally a team Agents change their autonomy, roles, coordination strategies • A String Quartet is a team Well organized and practiced
Team- Phil Cohen, et al: Shared goal and shared mental states Communication in the form of Speech Acts is required for team formation Steps to become a team: 1. Weak Achievement Goal (WAG) relative to q and with respect to a team to bring about p if either of these conditions holds: • The agent has a normal achievement goal to bring about p; that is, the agent does not yet believe that p is true and has p eventually being true as a goal. • The agent believes that p is true, will never be true, or is irrelevant (that is, q is false), but has as a goal that the status of p be mutually believed by all the team members. 2. Joint Persistent Goal (or JPG) relative to q to achieve p just in case 1. They mutually believe that p is currently false; 2. They mutually know they all want p to eventually be true; 3. It is true (and mutual knowledge) that until they come to mutually believe either that p is true, that p will never be true, or that q is false, they will continue to mutually believe that they each have p as a weak achievement goal relative to q.
Team- Phil Cohen, et al • Requiring Speech Act Communication is too strong • Requiring Mutual Knowledge is too strong • Requiring agents to remain in a team until everyone knows about the team-qualifying condition is too strong
Team- Munindar Singh <agents, social commitments, coordination relationships> Social commitments: <debtor, creditor, context, discharge condition> Operators: Create, Discharge, Cancel, Release, Delegate, Assign Coordination relationships about events: e is required by f e disables f e feeds or enables f e conditionally feeds f …
Team- Michael Wooldridge With respect to agent i’s desires there is potential for cooperation iff: 1. there is some group g such that i believes that g can jointly achieve ; and either 2. i can’t achieve in isolation; or 3. i believes that for every action that it can perform that achieves , it has a desire of not performing . i performs speech act Form. Team to form a team iff: 1. i informs team g that the team J-can ; and 2. i requests team g to perform Team g is a Pre. Team iff: 1. g mutually believe that it J-can ; 2. g mutually intends
Team- Michael Wooldridge • Onset of cooperative attitude is independent of knowing about specific individuals • Assuming agent knows about g is hard too simplistic • Requiring Speech Act Communication is too strong • Requiring Mutual Knowledge is too strong
![Team- [Hexmoor and Beavers 2001] Necessary components of a team: • Ability • Objective](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-86.jpg "Team- [Hexmoor and Beavers 2001] Necessary components of a team: • Ability • Objective")
Team- [Hexmoor and Beavers 2001] Necessary components of a team: • Ability • Objective • Awareness • Attitude: Cooperation and Responsibility Working Conditions: • Agent’s anticipatory power of team sensing and acting • Laws, Norms, Conventions, Commitments Motivations for team formation: • Shortcomings in ability • Efficiency • Failure/fault tolerance
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Game-Theoretic and Decision-Theoretic Agents n Utilitarian rationality: every agent has an utility function representing agent’s preferences over different alternatives. Every agent is concerned with maximizing his expected utility n Decision theory deals with situations in which one or more agents must make choices among given alternatives (Anatol Rapoport). Every choice has outcomes. Agents have preferences for different outcomes. n Decision making: u under certainty. under risk. under uncertainty. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Game-Theoretic and Decision-Theoretic Agents (cont. ) n Utility theory can be used in both decision making under risk (where the probabilities are explicitly given) and in decision making under uncertainty (where the probabilities are not explicitly given). n Three different approaches: Descriptive approach. u Normative. u Prescriptive approach. u SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Foundations of Utility: Lottaries Let A 1 and A 2 be any two events. Let 0 p 1. Then by (p. A 1, (1 -p)A 2) we mean the lottery which has the two possible outcomes A 1 and A 2 with probabilities p and 1 -p respectively. A B A is preferred or indifferent to B A is indifferent to B SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Axioms of Utility n Theorem [Luce and Raiffa, 1957].](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-91.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Axioms of Utility n Theorem [Luce and Raiffa, 1957].")
Hexmoor&Braynov Multiagents: Formal and Economic Axioms of Utility n Theorem [Luce and Raiffa, 1957]. If the preference relation is complete, transitive, continuous, and monotonic, then there exists an utility function U, such that: L L*, iff U(L) U(L*) L L*, iff U(L)=U(L*) OR n Theorem [Fishburn, 1970]. If the preference relation is a weak order on X and X/ is countable, then there is a real -valued function U on X such that: L L*, iff U(L) U(L*) L L*, iff U(L)=U(L*) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Utility of Money n The St. Petersburg Paradox (Daniel Bernoulli): How much would you pay to play the following game? A fair coin is continually tossed until it lands on heads. If the coin lands on heads on the nth throw, you receive 2 n dollars. EU=2. (1/2)+4(1/4)+8(1/8)+…=1+1+1+…. . n Risk attitudes: u u u Risk averse (concave utility function) U(p. A 1+(1 -p)A 2) > p. U(A 1)+(1 -p)U(A 2) Risk seeking (convex utility function) U(p. A 1+(1 -p)A 2) < p. U(A 1)+(1 -p)U(A 2) Risk neutral (linear utility function) U(p. A 1+(1 -p)A 2) = p. U(A 1)+(1 -p)U(A 2) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Decision Making Using Infinite Beliefs n Infinite beliefs arise naturally. An agent’s optimal decision depends on what he believes the other bidders will do, which in turn depends on what he believes the other bidders believe about him, and so on. This leads to infinite regress of beliefs. n Problems: u How to represent infinitely nested beliefs u How to reason and make decision with infinite beliefs. n Our solution [Brainov, Sandholm, 2000]: u We generalize the principle of backward induction to the case of infinite belief trees. u We identified a class of infinite belief trees that allow finite representation. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Cutting Infinite Belief Trees [Gmytrasiewicz and Durfee, 95] t 1 p 1 -p t 1 q t 1 . . . t 2 1 -q q t 2 t 1 . . . 1 -q t 2 . . . No information SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Backward Induction Step 4: Solve the first level i’s level p Step 3: Solve the second level 1 -p t 2 t 1 p t 1 Step 2: Solve third level p t 1 Step 1: Solve the bottom level SAC 2002 Tutorial 1 -p t 2 1 -p p t 2 t 1 p t 1 -p p t 2 t 1 j’s level 1 -p t 2 1 1 -p p t 2 t 1 i’s level 1 -p t 2 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Representation of Infinite Belief Trees with Finite Graphs n Two nodes of v 1 and v 2 of belief tree i are identical iff: · they are labeled with the same tk, tk T, · v 1 and v 2 are both on an even or an odd-numbered level of reflection, · v 1 and v 2 have the same successors, · every two arcs starting at v 1 and v 2, that point to the same successor, are labeled with equal probabilities. n An elementary contraction of a graph G is obtained by identifying two identical nodes v 1 and v 2 by removing v 1 and v 2, and by adding a new node v adjacent to those nodes to which v 1 and v 2 were adjacent. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Reasoning on Graphs n A graph G is contractible to a graph G’ if G’ can be obtained from G by applying elementary contractions. n Proposition: If a belief tree is contractible to a graph, then the graph is pointed and accessible. n Proposition: For every accessible pointed graph G there exists a belief tree that is contractible to G. n A strategy labeling is balanced if the strategy associated with each node is a best response to the strategies associated with the successor nodes, given the probabilities assigned to the successors. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Balanced Strategy Labeling for Infinite Tree t 1 S 1 i’s level of reflection p j’s level of reflection SAC 2002 Tutorial 1 -p t 1 S 2 p i’s level of reflection t 1 S 4 . . . 1 -p t 2 S 3 p S 5 t 2 t 1 S 6 . . . 1 -p S 7 t 2 . . . 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Infinite Belief Trees (cont. ) Balanced Strategy Labeling for Infinite Graph S 1 Agent i p 1 -p S 3 S 2 t 1 Agent j t 2 p p p Agent i SAC 2002 Tutorial S 4 t 1 p 1 -p 1 -p t 2 S 5 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example: Auction Analysis n English (first-price open cry). An ascending auction where the price is successively raised until one bidder remains. That bidder wins the object and pays his final price. n Dutch. A descending auction where the auctioneer starts at a very high price and then lowers the price gradually. The first bidder who stops him takes the object at the current price. n First-price sealed-bid auction. Each bidder independently submits a bid without knowing others’ bids. The highest bidder wins and pays his bid. n Second-price sealed-bid auction. Each bidder independently submits a bid without knowing others’ bids. The highest bidder wins and pays the amount of the second highest bid. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example (cont. ) n Revenue equivalence theorem [William Vickrey, 1961]: The firstprice sealed bid, second-price sealed bid, English and Dutch auctions are all optimal selling mechanisms provided that they are supplemented by optimally set reserve price. n Simple Auction Setting: u Isolated auction for a single indivisible object with two risk neutral bidders. u Two possible valuations of the object: t 1 and t 2 (t 1<t 2). t 1 and t 2 are independent random variables with objective distribution =(1/2, 1/2). u Each bidder knows his own valuation, but is uncertain about his rival’s evaluation. u is common knowledge. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example (cont. ) Revenue Equivalence Theorem fails: n Proposition. When there does not exist common knowledge about private beliefs, the revenue equivalence theorem ceases to hold. The bidder’s expected utility is different in the first price sealed bid auction and Vickrey auction. Agent i p 1 -p 1/2 Agent j t 2 t 1 Agent j p 1/2 t 1 SAC 2002 Tutorial t 2 t 1. . . t 2 1/2 1/2 t. 1. . 1 -p t. 2. . Agent i t 1 1/2 1/2 1/2 t 2 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Agent as a member of a group. . . agent honors handles partakes roles obligations goals specifies contains values (terminal goals) SAC 2002 Tutorial member of plans norms relies on partakes shares group 3/13/02
Values • "value" might mean: (a)assessment of usefulness of an object or action relative to a purpose, I. e. , "(instrumental) evaluations", E. g. , ="this knife is good for chip carving ", (b) absolute assessment of desirability of something, I. e, “principles”, E. g. , "honesty is good" • Adding value to an agent enables it to generate internal desires as well as adds a level of behavior predictability for other agents.
Norms • Involve two or more agents. Each agent understands and shares them. • Agents have power to not choose them. • There is no direct rational account of them available to the agents. • The bearer experiences an implicit or an explicit sanction or rewards for adoption.
Norms Norm = (O, R, G, A) · O is the content of the norm set. These can at least one goal to do or at least one state that it may avoid. · R is the sanction that may result from not following the norm. · G is the agent’s goal that invokes the norm set when the agent chooses to consider other agents • A is a set of mental stances along with degrees for each. The mental stances are characterized by using the notions of belief, desire, and intention. The degrees of mental stances forms a required pattern against which we will match the agent’s actual mental stances.
Obligations • Obligations capture all forms of social influence. • Obligations have a strong deontological and motivational senses (more so than norms) • Obligations are frequently assumed to have penalties associated with the failure to meet the obligation. We make no such assumption; some obligations may have sanctions and some may not.
Dependence and Control • An agent a may depend on another agent b for performing an action when agent b is obligated (with consent) to agent a for performing the action. . More to come • An agent a controls an agent b in a given domain when (a) agent b adopts goals set by agent a in the domain, and (b) agent a monitors agent b about the goal and gives it feedback • If furthermore, agent b incorporates the feedback from agent a, the control is master-slave.
Responsibilities • There are several types of responsibility: (a)Responsibility to concerns an agent’s obligation to perform an action. (b)Responsibility for concerns an agent’s obligation to see that a state of affairs obtains. (c) Responsibility about is the agent’s obligation to behave in accordance with its principles, which is general, abstract, and typically with respect to an agent’s immutable values.
![Responsibilities, CAST project [Yen, et al. 2001] • Agents are represented as nodes of](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-111.jpg "Responsibilities, CAST project [Yen, et al. 2001] • Agents are represented as nodes of")
Responsibilities, CAST project [Yen, et al. 2001] • Agents are represented as nodes of a graph. • One type of labeled directed edge is between two agents (A t B), and it represents that A delegates t to B or conversely B is responsible to A with respect to t. • The delegation relationships is non-reflexive, antisymmetric, and transitive. The transitive property can be used to establish implied relationships.
Roles • Several agents can adopt it individually, independently, and concurrently. One agent may adopt several simultaneously. Several agents may adopt it as a group. In general we will call this the adopter. • It is meaningful in the social context of other agents including (a) the adopter’s relationship to other agents and groups, (b) the agent’s mental attitudes about the social relationships, and (b) the available norms including obligations and responsibilities. • There are typical capabilities associated with the adopter. If the adopter is loses these abilities then the efficacy of the role is jeopardized.
Roles • Networks of roles are more clearly seen in role-based access control. • Role hierarchy and role grouping are useful for selecting subsequent roles [Moffett and Lupu, 1999, Na and Cheon, 2000].
The big picture Norms Values Obligationsab (i. e. , responsibility) Dependenceba Delegationba Powerab Trustba Autonomyb + Autonomya Controlab
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction Self-Interest and Benevolence in Multiagent Interaction [Brainov, 1994, 1999] n Limitations of self-interested behavior: u Free-rider problem u Not always efficient allocation of tasks and resources n Benevolent agents are important when u An agent represents preferences of non-selfish users u An agent represents collective users u Several agents represent one user n Malevolent agents are important in adversarial environments n Model of social attitudes SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans n Joint Plans: N={1, 2, …, n} (p 1, p 2, …, pn) p. N-i - Ui - the set of all agents a joint plan the plan of the group of agents N-{i} utility of agent i (p 1, p 2, …, pn) = (p. N-i, pi) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans Models of Power and Dependence n Social dependence network [Castelfranchi, Conte, Sichman, Demazeau] u Only two types of dependence (resource and action dependence) u Individual dependence (an agent depends on another agent n Dependence as an element of power [Brainov, 1998; Brainov&Sandholm, 1999] u Decision-oriented model of dependence u Group dependence (an agent depends on a group, group depends on an agent, group depends on a group) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Elements of Power Means Base Cost Power Amount SAC 2002 Tutorial Scope 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans Individual Dependence n An agent i depends on another agent j in a joint plan if agent j can gain at the expense of agent i agent by deviating from the plan. n Formally: In a joint plan (p. N-j, pj) agent i depends on agent j if there exists plan p*j such that for every plan p’N-j of the other agents: Ui(p’N-j, p*j) < Ui(p. N-j, pj), Uj(p. N-j, p*j) Uj(p. N-j, pj). The plan p*j is said to be gainful for agent j at the expense of agent i. To indicate that the gainful plan depends on the initial plan (p. N-j, pj), we denote it by pjgain(i, p. N-j, pj). SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example j i Help! j SAC 2002 Tutorial i 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example I can help you. I can also help you. j k i SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans A Group Depends on an Agent [Brainov and Sandholm, 99] A group S depends on an agent j if every member of S depends on j for the same plan p*j. n Formally: In a joint plan (p. N-j, pj) a group of agents S, S N-j, depends on agent j if there exists a plan p*j such that for every plan p’N-j of the other agents: Uk(p’N-j, p*j) < Uk(p. N-j, pj) for every k S Uj(p. N-j, p*j) Uj(p. N-j, pj). The plan p*j is said to be gainful for agent j at the expense of group S. To indicate that the gainful plan depends on the initial plan (p. N-j, pj), we denote it by pjgain(S, p. N-j, pj). SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example We have to move the block. I can help you. j SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans An Agent Depends on a Group n An agent i depends on a group S, if u u n i depends on every member of the group (given that the rest of the group do not change their plans), the group S is minimal. Formally: In a joint plan (p. N-j, pj) agent i depends on a group of agents S, i S, if (i) (group requirement) agent i depends conditionally on every agent k, k S, with the tacit consent of the group S-k; (ii) (minimality requirement) for every agent k, k S, every conditionally gainful plan pkgain(i, p. N-S, p. S-k, pk/p. S-k) and every plan p’N-S, it holds that every agent m, m S-k, can gain without harming agent i in the plan (p’N-S, p. S-k, pkgain(i, p. N-S, p. S-k, pk/p. S-k)) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example j i k j k Help! i SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example j k i SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Power and Dependence in Multiagent Plans n A joint plan is individually stable if no single agent can deviate from the plan. n A joint plan is coalitionally stable if no group of agents can deviate from the plan in a way that benefits all its members. n Proposition. Every Pareto optimal joint plan which is based on reciprocal dependence is individually and coalitionally stable. n Proposition. If in a Pareto optimal joint plan (p. S, pj): (i) p. S is based on reciprocal dependence, (ii) group S depends on agent j, (iii) agent j depends on some agent i, i S, then the plan (p. S, pj) is individually and coalitionally stable. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Attitudes Agent typology Altruistic SAC 2002 Tutorial Self-interested Malevolent 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Benevolent Agents n. An agent is benevolent towards other agents if its utility function depends positively on the utility functions of other agents: n Formally: An agent a is benevolent towards the agents 1, …, k iff: Ua=Ua(Xa, U 1(X 1), …, Uk(Xk)) For all j=1, …, k Here Xa represents the benefits of agent a. U 1, …, Uk are utility functions of agents towards whom agent a is altruistic. X 1, …, Xk represent the benefits of these agents. n. Example: agents with a common goal, agents that serve one user, team of agents, etc. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Self-Interested Agents n n An agent is self-intersted if its utility function does not depend on the utility functions of other agents: Formally: An agent i is self intersted iff: Ui=Ui(Xi, U 1(X 1), …, Uk(Xk)) For all j=1, …, k Here Xi represents the benefits of agent i. U 1, …, Uk are utility functions of other agents. n Example: agents that serve different users, etc. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Malevolent Agents n. An agent is malevolent towards other agents if its utility function depends negatively on the utility functions of other agents: n Formally: An agent m is malevolent towards the agents 1, …, k iff: Um=Um(Xm, U 1(X 1), …, Uk(Xk)) For all j=1, …, k Here Xm represents the benefits of agent m. U 1, …, Uk are utility functions of agents towards whom agent m is malevolent. n Example: agents with antagonistic goals, rivals, competitors, virtual battlefield, etc. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Degree of Attitude degree of attitude ij of agent i towards agent j is defined as follows: n. The We call an agent i: u Self-biased, if ij<1; u Other-biased, if ij>1; u Neutral, if ij=1. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Derived Social Attitudes n n n If agent i has a social attitude towards agent j and agent j has a social attitude towards agent k, then the combined effect of these two attitudes is called a derived attitude of agent i towards agent k. Proposition. If ij is the degree of attitude of agent i towards agent j and jk is the degree of attitude of agent j towards agent k, then the degree of derived attitude, ik, of agent i towards agent k is: ik= ij jk Proposition. If ij is the degree of attitude of agent i towards agent j, jk is the degree of attitude of agent j towards agent k, and ik is the degree of attitude of agent i towards agent k, then the degree, *ik, of the final derived attitude of agent i towards agent k satisfies the following condition: *ik= ij jk+ ik SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Example friends enemies ? enemies SAC 2002 Tutorial friends 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Optimal Group Attitudes n Proposition. Let Ui(p)=worthi(gi)+ ijworthj(gj)-costi(p)- ijcostj(p) Uj(p)= jiworthi(gi)+worthj(gj)- jicosti(p)-costj(p) n where p is a joint plan of agents i and j, ij, ji are constants and ij>0, ji>0. If the cost functions of both agents are differentiable on the set of all joint plans P, and if there exists an equilibrium joint plan p* P, then: ij ji=1 Example: Conflict of attitudes: If one agent wants to help another agent, the first one has to be benevolent enough to provide help and the second agent has to be selfish enough to receive help. If both agents are other-biased, then both of them will want to help and nobody will want to receive help. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Solving Social Situations 1/2 -1/3 3 Incomplete situation 1/2 -1/3 2 -3 3 1/3 Possible completion SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Socially Responsible Agents [Jennings and Campos, 97] Principle of Social Rationality: If a member of a responsible society can perform an action whose joint benefit is greater than its joint loss, then it may select that action. D(M, a) - W(M, a) d(M, a) -w(M, a) n Members may perform actions for which their member benefit is less than their member loss, if the society gets more in total than it loses. n Members may perform actions which bring them personal benefit, but which are detrimental to the overall society. This is possible if the member benefit is greater than the societal loss. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Helping Agents [Cesta, Miceli and Rizzo, 99] n Lonely agents: ignore one another. There is no interaction among them. Their goal is always to individually find food. n Social agents: in case of danger, their goal generator activates the goal of looking for help; when hungry, their goal is to find food; and finally, in case of normal state, if there any visible needy agents, the goal of giving help is activated; otherwise, they go on looking for food. In an experimental environment social agents overperform lonely agents. SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Probabilistic reciprocity [Sen, 96] Adaptive,](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-140.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Probabilistic reciprocity [Sen, 96] Adaptive,")
Hexmoor&Braynov Multiagents: Formal and Economic Social Interaction (cont. ) Probabilistic reciprocity [Sen, 96] Adaptive, probabilistic, reciprocity-based policy for deciding which other agent to cooperate with. n Allows agents to initiate cooperative relationships. n Use a mechanism to compare cooperation costsallow agents to be inclined to help other agents with whom there is a favorable balance of help. n Flexible adjustment of help-giving behavior based on current work-load. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov n Reasons for trust: u u u n Time asymmetry: some agents have to accomplish their part of the transaction before other agents have taken actions. Lack of power: an agent does not have the power to control actions of other agents. Inability to conclude perfect contracts: contract are usually incomplete or indefinite. Trust as: u u n Trust Multiagents: Formal and Economic Expectation of partner’s competence Expectation of partner’s benign intent Models of trust: u u u Castelfranchi, Falcone [1999, 2000] Marsh [1994] Brainov, Sandholm [1999, 2001] SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) Contracting Problem The seller produces some commodity and sells it to the buyer. The transaction is secured by a contract stating the quantity and the selling price. - the level of trustworthiness of the buyer - the level of trustworthiness of the seller In our case =1, 0 1. The seller is not completely trustworthy. - the buyer’s estimate of - the seller’s estimate of SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) Optimal Trust [S. Brainov, T. Sandholm , 1999] n Proposition. If = , then the quantity exchanged maximizes the function the social welfare. n Proposition. The quantity maximizing the social welfare is the maximum possible output. n Proposition. When the trust placed matches trustworthiness ( = ) the seller and the buyer maximize their utilities. n The social welfare, the volume of trade and agents’ utilities are maximized if the buyer’s trust matches the seller’s trustworthiness. It is NOT necessary for the seller to be completely trustworthy SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) Improving Trustworthiness by Advanced Payments Proposition. If < and the agents choose an advanced delivery contract, then the quantity exchanged maximizes the social welfare, the buyer makes a complete advance delivery, and the volume of trade is maximal. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) Incentive Compatible Mechanism for Trust Revelation n If the buyer asks the seller about his level of trustworthiness, the seller may lie. n Incentive compatibility: the seller has sufficient incentives to reveal truthfully his level of trustworthiness. n The mechanism: u First, the seller declares his level of trustworthiness . u Using the seller’s declaration , the buyer chooses the quantity to be exchanged q=q( ) u The price P is exogenously determined by market forces. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) n Proposition. If the seller’s cost function is twice differentiable and convex, there exists exactly one quantity function q( ) that makes the seller reveal truthfully his level of trustworthiness. n Proposition [Braynov, 2001]. If the seller’s cost function is convex and the buyer’s value function is concave, then the mechanism is individually rational for both the buyer and the seller (i. e. , they will benefit by participating in the mechanism). SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Trust (cont. ) Advantages of the Mechanism It does not require the estimation of other agents’ trustworthiness. For example, agents do not need to reason, plan and search for estimates of other agents’ trustworthiness. n It eliminates the need to speculate on other agents’ intentions and beliefs. This could simplify individual decision-making and save some deliberation costs. n It may reduce the cost of trust management. Since agents are reporting truthfully their levels of trustworthiness, recording, analyzing and aggregating reputation information is simplified. n It could eliminate many market failures caused by the lack or inaccuracy of information. Truthful estimates of the agents’ trustworthiness are available for every transaction. SAC 2002 Tutorial n 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Communication Speech Act Theory [Searle, 70; Austin, 62] Natural](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-149.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Communication Speech Act Theory [Searle, 70; Austin, 62] Natural")
Hexmoor&Braynov Multiagents: Formal and Economic Communication Speech Act Theory [Searle, 70; Austin, 62] Natural language communication consists of speech acts such as requests, suggestions, commitments, replies. etc. Utterances: propositional utterances illocutionary utterances perlocutionary utterances intention to affect behavior intention to interact reference to the world SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Knowledge Query and Manipulation Language (KQML) (KQML-performative : sender <word> : receiver <word> : language <word> : ontology <word> : reply-with <expression> : in-reply-to <expression> : content <expression>. . . ) SAC 2002 Tutorial semantics 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) n n n KQML communication could be asynchronous or synchronous. KQML is a communication protocol for both agents and application programs. KQML separates the domain semantics from the semantics of the communication protocol. Both the sender and the receiver have to understand KQML and use the same ontology. The network infrastructure is not a part of KQML: agents cannot locate one another. They use facilitators. SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Knowledge Interchange Format (KIF) [Genesereth, 1991]](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-152.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Knowledge Interchange Format (KIF) [Genesereth, 1991]")
Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Knowledge Interchange Format (KIF) [Genesereth, 1991] KIF is a prefix version of first order predicate calculus with extensions to support nonmonotonic reasoning and definitions. (=> (bird joe) (can_fly joe)) (interested john '(can_fly ? x)) SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Ontologies An ontology is a formal, explicit specification of a shared conceptualization [Gruber, 93]. n Domain ontologies n Metadata ontologies n Generic or common sense ontologies n Representational ontologies SAC 2002 Tutorial 3/13/02
![Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Ontology languages n CYC [Lenat&Guha, 90]:](https://present5.com/presentation/1f7cc1e4d2ea382a9ec9dde6addcd14c/image-154.jpg "Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Ontology languages n CYC [Lenat&Guha, 90]:")
Hexmoor&Braynov Multiagents: Formal and Economic Communication (cont. ) Ontology languages n CYC [Lenat&Guha, 90]: based on the first-order predicate calculus. Uses constants, predicates, variables, formulas, functions and quantifiers. Quantification is also allowed over predicates, functions, arguments and formulas. n Ontolingua [Gruber, 93]: this is an object-oriented approach. (define-relation name (? A 1 ? A 2) : def (KIF fornula)) n Frame logic [Kifer et al. , 95]: a language for specifying object-oriented databases. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security n Host security: protecting the hosting computer, its services, data and its reputation n Mobile agent security: protecting agent code, state, data and reputation. n Security threats: n Disclosure of information n Denial of service n Corruption of information n Masquerading n Unauthorized Access n Repudiation n Eavesdropping n Alternation SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security (cont. ) Home Platform Agent Platform Network Agent Platform SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security (cont. ) An agent attacking an agent platform An agent platform attacking an agent Thread categories: An agent attacking another agent on the same platform Other entities attacking an agent SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security (cont. ) Protection Fault Isolation: Application modules are isolated into distinct fault domains. Sandboxing. n Software-Based Safe Code Interpretation: commands considered harmful can be either made safe for or denied to an agent. n Signed Code: signing code or data with digital signature. Microsoft Authenticode. n State Appraisal: the author produce and digitally signs state appraisal functions. n Path histories: to keep authenticatable record of the prior platforms visited by an agent. n Proof Carrying Code: the author produces a formal proof that the program possesses safety properties previously stipulated by the consumer. n SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security (cont. ) n Proof Carrying Code: the author produces a formal proof that the program possesses safety properties previously stipulated by the consumer. n Partial result encapsulation: encapsulation of the results of an agent’s actions at each platform visited. n Mutual Itinerary Recording: when moving between agent platforms, an agent conveys the last platform, current platform, and the next platform information to the cooperating peer agent. n Replication: rather than a single copy of an agent performing a computation, multiple copies of the agent are used. n Shared secrets and interlocking. n Execution tracing: faithful recording of the agent’s behavior during its execution on each platform. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Security (cont. ) n Environmental key generation: upon encountering an environmental condition a key is generated which is used to unlock some executable code cryptographically. n Computing with encrypted functions (executable encrypted functions): the platform executes a program implementing an encrypted function without being able to discern the original function. n Using dummy items and functions. n Watermarks and steganographic techniques. n Smart-card solutions. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning agent: an agent that can improve its behavior by analyzing its own and others’ experience. Agent’s abilities: n An agent is able to sense the environment to some extent. n An agent is able to take actions that affect the state of the environment. Important elements: n Learning element: responsible for making improvements. n Performance element: responsible for selecting actions. n Performance standard: fixed, beyond the control of the agent. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Environment: n Accessible: the agent percept at each step will identify the state it is in. n Inaccessible or partially accessible Supervised learning: there is a supervisor, who provides both the actions and their results. Reinforcement learning: the feedback is in the form of reward(reinforcement). An agent learn its utility functions on states. Every agent tries to maximize its (expected) utility. Q-Learning: an agent learns an action-value function giving the expected utility of taking a given action. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Rewards: n n Can be received only in terminal states. Can be received in any state. Can be components of actual utility. Can be hits as to the actual utility. n Centralized learning: the learning process is executed by a single agent and does not require any interaction with other agents. n Decentralized learning: several agents or groups of agents are engaged in the learning process. Agents may have the same or different learning goals. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Layered learning [Stone P. , 2000; Stone P. Veloso M. , 2000] n n Layered learning is designed for domains that are too complex for learning a mapping directly from the input to the output representation. Layered learning uses a bottom-up incremental approach to hierarchical task decomposition. Learning occurs separately at each level. Each learning layer directly affects the learning at the next layer. A learned subtask can affect the subsequent layer by: constructing the set of training examples; providing the features used for learning; or pruning the output set. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Layered learning in robotic soccer n Layer 1: An individual skill: ball Interception. Intercept opponent shots or passes as well as receive passes from teammates n Layer 2: Multiagent behavior: pass evaluation. Agents (both teammates and opponents) are equipped with the previously learned ball-interception behavior. n Layer 3: Collaborative and adversarial team behavior: pass selection. The input space is drastically reduced the input space with the help of the previously learned decision tree. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Team-Partitioned Opaque-Transition RL [Stone P. , Veloso M. , 1999] n Team Partitioning: Each agent explores a separate partition of the state space without any knowledge of state values in other partitions. Agents are learning in parallel. n Opaque-transition setting: since teammates and opponents can affect the outcome of an action, state transitions are opaque to an agent. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) State generalization: every state is generalized to a feature vector v using the state generalization function f: S V. n Value function learning: the feature vector v is used to estimate the expected reward for taking an action in a state – Q(V, A) R. Q(v, a)=Q(v, a)+ (r-Q(v, a)) n Action selection: an action is chosen for execution and its reward is used to update Q. An action is chosen either randomly when exploring, or according to maximum Q-value when eploiting. n SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Improving Opponent Models into Adversary Search [Carmel and Markovitch, 96] S : S 2 S : S S f d SAC 2002 Tutorial -the set of all possible states - the successor function - the opponent model - utility function - a depth limit 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Let M 0(<f 0>, d)(s) - regular minimax algorithm M 0(<f 0>, d)(s)=M(s, d, f 0, M 0(<-f 0>, d-1)) M 1(<f 1, f 0>, d)(s)=M(s, d, f 1, M 0(<f 0>, d-1)) …………………. . Mn(<fn, . . . , f 0>, d)(s)=M(s, d, fn, Mn-1(<fn-1…. f 0>, d-1)) Definition: A player is a pair as follows: • Given a utility function f, P=(f, NIL) is a player with a modeling-level 0. • Given a utility function f and a player O (with modelinglevel n-1), P=(f, O) is a player with a modeling-level n. Different players: (f 0, NIL), (f 1, (f 0, NIL)), (f 2, (f 1, (f 0, NIL))), ……. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Game Tree a c b d h f 2= 8 f 1= 6 f 0= 4 SAC 2002 Tutorial e i f 2= -4 f 1= 6 f 0= -8 j f 2= 4 f 1= -8 f 0= 10 f k f 2= 7 f 1= -7 f 0= 3 l f 2= -6 f 1= 7 f 0= -4 m f 2= 1 f 1= -2 f 0= 4 g n f 2= 10 f 1= -4 f 0= 4 o f 2= 2 f 1= 0 f 0= 6 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Learning in market environments [Vidal, Durfee, 96] Several buyers and sellers. These agents exchange goods by paying some price psg. When a buyer wants to buy a good g, he will advertise this fact. Each seller that sells that good will give his bid in the form of a price psg. The buyer will pick one of these and will pay the seller. The seller will then return the specified good. Vbg(p, g) - value function, returns the value that the buyer b assigns to that particular good at that particular price and quality. psg-csg - the seller's profit SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Multiagent Learning (cont. ) Types of agents: 0 -level agents: they do not model the behavior of the other agents. 1 -level agents: they model other agents as 0 -level agents. 2 -level agents: they model other agents as 1 -level agents n If buyers are 0 -level agents and sellers are 1 -level agents, seller can pretend to be high-quality goods sellers by bidding high prices and thus obtain substantial profits at the expense of the buyers. n If the buyers are 1 -level agents, they learn to buy from sellers who can provide them with the highest value. n If the buyers and the sellers are 1 -level agents, 1 -level sellers suffer, because they assume buyers are 0 -level agents and try to over-price their goods. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Current Trends Pervasive and emerging agent applications: agent mediated e-commerce, emotional agents, embodied agents, virtual characters, conversational agents, etc. n n Standardization efforts: FIPA. n New Initiatives: semantic web initiative. Agent tournaments: Robo. Cup, Trading Agent Competition. n SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Outline 1. History and perspectives on multiagents 2. Agent Architecture 3. Agent Oriented Software Engineering 4. Mobility 5. Autonomy and Teaming 6. Game Theoretic and Decision Theoretic Agents 7. Social attitudes: Values, norms, obligations, dependence, control, responsibility, roles 8. Benevolence, Preference, Power, Trust. 9. Communication, Security 10. Agent Adaptation and Learning 11. Trends and Open questions 12. Concluding Remarks SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Concluding Remarks n There are many uses for u Agents u Agent-based Systems u Agent Frameworks n Many open problems area available u Theoretical issues for modeling social elements such as autonomy, power, trust, dependency, norms, preference, responsibilities, security, … u Adaptation and learning issues u Communication and conversation issues SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic Further Explorations n n DAI-List@engr. sc. edu Agents. umbc. edu http: //www. Agent. Link. org/ http: //www. multiagent. com/ SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic References §Austin J. How to do Things with Words. Calderon, Oxford, 1962. §Braynov S. . Incentive Compatible Mechanism for Trust Revelation. IJCAI’ 01 Workshop on Economic Agents, Models and Mechanisms, 2001. §Brainov S. , Sandholm T. Reasoning about Others: Representing and Processing Infinite Belief hierarchies, ICMAS’ 2000. §Brainov S. , T. Sandholm T. Contracting with Uncertain Level of Trust. In Proceedings of the ACM Conference on E-Commerce, 1999. §Brainov S. , T. Sandholm T. Power, Dependence and Stability in Multiagent Plans. In Proc. Of AAAI’ 99, pp: 11 -16, 1999. §Brainov S. Altruistic Cooperation Between Self-Interested Agents, European Conference on AI, pp: 519 -523, 1996. §S. Brainov S. The Role and the Impact of Preferences on Multiagent Interaction, ATAL’ 99, 1999. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic References (cont. ) § Carmel D. , Markovitch S. Incorporating Opponent Model into Adversary Search, In Proc. Of AAAI’ 96, pp: 120 -125, 1996. § Castelfranchi C. , Falcone R. , (2000), Trust and Control: A Dialectic Link, Applied Artificial Intelligence journal, Special Issue on "Trust in Agents" Part 1, Castelfranchi C. , Falcone R. , Firozabadi B. , Tan Y. (Editors), Taylor and Francis 14 (8), 2000. § Castelfranchi, C. , Falcone, R. (1999). The Dynamics of Trust: from Beliefs to Action, Autonomous Agents Î99 Workshop on "Deception, Fraud and Trust in Agent Societies", Seattle, USA, May 1, pp. 41 -54. § Cesta A. , Miceli M. , Rizzo P. Proceedings of the International Workshop on the Design of Cooperative Systems. Proceedings of the International Workshop on the Design of Cooperative Systems, 1995. § Genesereth M. Knowledge Interagent Format. In Proc. Of KR-91, pp: 238 -249, 1991. § Gmytrasiewicz, P. , Durfee, E. A Rigorous, Operational Formalization of Recursive Modeling, In Proceedings of the First International Conference on Multi-Agent Systems, pages 125 -132, 1995. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic References (cont. ) § Gruber T. A Translation Approach to Portable Ontology Specification, Knowledge Acquisition, 5: 199 -220, 1993. § Hexmoor, H. , Holmback H. , and Duncan, L. Detecting, Expressing, and Harmonizing Autonomy in Communication Between Social Agents, 2001 AAAI spring symposium on Robust Autonomy, Stanford, AAAI press. § Hexmoor H. and Harry Duchscherer, H. Shared Autonomy and Teaming: A preliminary report. In Proceedings of Workshop on Performance Metrics for Intelligent Systems, NIST, 2000, Washington, DC. § Hexmoor, H. A Cognitive Model of Situated Autonomy, In Proceedings of PRICAI-2000 Workshop on Teams with Adjustable Autonomy, Australia. § Hexmoor, H. Weakly Dependent Agents May Rely on Their Luck or Collaboration: A Case for Adaptation, In AISB-2001, York, UK. § Hexmoor, H. and Duchscherer, H. Efficiency as Motivation for Teaming, In Proceedings of FLAIRS 2001, AAAI press. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic References (cont. ) § Hexmoor, H. and Beavers, G. Towards Teams of Agents, In Proceedings of the International Conference in Artificial Intelligence, H. R. Arabnia, (ed), (IC-AI'2001), Las Vegas, CSREA Press. § Hexmoor H. and Zhang, X. Norms, Roles, and Simulated Robo. Cup, In 2 nd workshop on norms and institutions in multiagent systems, (Agents 2001), Montreal, CA, ACM press. § Jennings N. , Campos J. Towards a Social Level Characterization of Socially Responsible Agents. in IEE Proceedings on Software. Engineering, 144 (1), 11 -25, 1997, § Fishburn P. Utility Theory for Decision Making. Wiley, New York, 1970. § Kifer M. , lausen G. , Wu J. Logical Foundations of Object-Oriented and frame-Based languages. Journal of the ACM, 42, 1995. § Lenat D. , Guha R. Building Large Knowledge-Based Systems. Addison-Wesley, 1990. § Luce R D, Raiffa H. Games and Decisions, Wiley, New York, 1957. § Marsh S. Formalising Trust as a Computational Concept. Ph. D Thesis, University of Stirling, April 1994 § Searle J. Speech Acts: An Essay in the Philosophy of Language. , Cambridge, 1970. SAC 2002 Tutorial 3/13/02
Hexmoor&Braynov Multiagents: Formal and Economic References (cont. ) § Sen S. Reciprocity: a foundational principle for promoting cooperative behavior among self- interested agents'' , in Proc. of the Second International Conference on Multiagent Systems, pp. 322 -329, 1996. § Sichman J. , Rosaria C. , Demazeau Y. , Castelfranchi C. A social reasoning mechanism based on dependence networks. In: Proceedings of the 11 th European Conference on Arti- cial Intelligence, 1994. § Stone P. , TPOT-RL Applied to Network Routing. Seventeenth International Conference on Machine Learning, 2000. § Stone P. , Veloso M. Layered Learning. Eleventh European Conference on Machine Learning, 2000. § Stone P. , Veloso M. Team-Partitioned, Opaque-Transition Reinforcement Learning, Agents’ 99, 1999. § Vickrey, W. Counterspeculation, Auctions, and Competitive Sealed Tenders, Journal of Finance 16, 8 -37, 1961. § Vidal J. , Durfee E. The Impact of Nested Agent Models in an Information Economy. In Proc. Of the ICMAS’ 96, pp: 377 -384, 1996. SAC 2002 Tutorial 3/13/02
Скачать презентацию SAC 2002 Tutorial Henry Hexmoor Svet Brainov University
1f7cc1e4d2ea382a9ec9dde6addcd14c.ppt