Agent Infrastructure Michael Luck University of Southampton UK

Agent Infrastructure Michael Luck University of Southampton, UK

Part I The Case for Agent-Oriented Software Engineering with slides borrowed from Nick Jennings

Remote Agent Experiment (RAX) § Deep Space One mission to validate technologies § AI software in primary command of a spacecraft

RAX § Comprises • planner/scheduler to generate plans for general mission goals • smart executive to execute plans • Mode identification and recovery to detect failures § Goals not pre-planned so more flexible § Tests include simulated failures § Tests in May 1999

Agents § § § Relatively new field (10 -15 years? ) Dramatic growth Popularity Increasing numbers of applications Multi-disciplinary Problems: • Agent backlash? • Sound conceptual foundation?

Agent-Based Computing: A New Synthesis for AI and CS Increasing number of systems viewed in terms of agents: • as a theoretical model of computation § more closely reflects current computing reality than Turing Machines • as a model for engineering distributed software systems § better suited than object-orientation, design patterns and software architectures • as a model for conceptualising and building intelligent entities § framework for unifying piecemeal specialisations 6

Which Perspective? § § Answer from many points of view from philosophical to pragmatic Proceed from standpoint of using agent-based software to solve complex, real-world problems

Software Development is Difficult § One of most complex construction task humans undertake “Computer science is the first engineering discipline ever in which the complexity of the objects created is limited by the skill of the creator and not limited by the strength of the raw materials. If steel beams were infinitely strong and couldn’t ever bend no matter what you did, then skyscrapers could be as complicated as computers. ” Brian K. Reid § True whatever models and techniques are applied “the essential complexity of software” Fred Brooks § Software engineering provides models & techniques that make it easier to handle this essential complexity 8

Software Development is Getting Harder § Shorter development lifecycles § More ambitious requirements § Less certain requirements • Greater scope for change § More challenging environments • Greater dynamism • Greater openness

Software Engineering: Continually Playing Catch Up § Better Models • components • design patterns • software architectures § Better Processes • light methods • heavier methods • interacting agents “Our ability to imagine complex applications will always exceed our ability to develop them” Grady Booch

The Adequacy Hypothesis Agent-oriented approaches can enhance our ability to model, design and build complex distributed software systems.

The Essence of Agent-Based Computing

Agent “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its design objectives” (Wooldridge) 13

Agent “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its design objectives” (Wooldridge) § control over internal state and over own behaviour 14

Agent “encapsulated computer system, situated in some environment, and capable of environment flexible autonomous action in that environment in order to meet its design objectives” (Wooldridge) § control over internal state and over own behaviour § experiences environment through sensors and acts through effectors 15

Agent “encapsulated computer system, situated in some environment, and capable of flexible autonomous action in that environment in order to meet its design objectives” (Wooldridge) § control over internal state and over own behaviour § experiences environment through sensors and acts through effectors § reactive: respond in timely fashion to environmental change § proactive: act in anticipation of future goals 16

Definitional Malaise “My guess is that object-oriented programming will be what structured programming was in the 1970 s. Everybody will be in favour of it. Every manufacturer will promote his product as supporting it. Every manager will pay lip service to it. Every programmer will practice it (differently). And no one will know just what it is. ” (Rentsch, 82) “My guess is that agent-based computing will be what objectoriented programming was in the 1980 s. Everybody will be in favour of it. Every manufacturer will promote his product as supporting it. Every manager will pay lip service to it. Every programmer will practice it (differently). And no one will know just what it is. ” (Jennings, 00)

Multiple Agents In most cases, single agent is insufficient • no such thing as a single agent system (!? ) • multiple agents are the norm, to represent: § natural decentralisation § multiple loci of control § multiple perspectives § competing interests 18

Agent Interactions § Interaction between agents is inevitable • to achieve individual objectives, to manage interdependencies § Conceptualised as taking place at knowledge-level • which goals, at what time, by whom, what for § Flexible run-time initiation and responses • cf. design-time, hard-wired nature of extant approaches paradigm shift from previous perceptions of computational interaction 19

Organisations Agents act/interact to achieve objectives: • on behalf of individuals/companies • part of a wider problem solving initiative underlying organisational relationship between the agents 20

Organisations This organisational context: • influences agents’ behaviour § relationships need to be made explicit – peers – teams, coalitions – authority relationships • is subject to ongoing change § provide computational apparatus for creating, maintaining and disbanding structures 21

A Canonical View Agent Interactions Sphere of influence Organisational relationships Environment (see also: Castelfranchi, Ferber, Gasser, Lesser, …. . )

Making the Case: Quantitatively “There are 3 kinds of lies: lies, damned lies and statistics” Disraeli

Making the Case: Qualitatively Software techniques for tackling complexity

Tackling Complexity § Decomposition § Abstraction § Organisation

Making the Case: Qualitatively Software techniques for tackling complexity Nature of complex systems

Complex Systems § Complexity takes form of “hierarchy” (Herb Simon) • not a control hierarchy • collection of related sub-systems at different levels of abstraction § Can distinguish between interactions among sub-systems and interactions within sub-systems • latter more frequent & predictable: “nearly decomposable systems” § Arbitrary choice about which components are primitive § Systems that support evolutionary growth develop more quickly than those that do not: “stable intermediate forms”

Making the Case: Qualitatively Software techniques for tackling complexity Agent-based computing Nature of complex systems

Making the Case: Qualitatively Software techniques for tackling complexity Degree of Match Nature of complex systems Agent-based computing

The Match Process 1. Show agent-oriented decomposition is effective way of partitioning problem space of complex system 2. Show key abstractions of agent-oriented mindset are natural means of modelling complex systems

The Match Process 1. Show agent-oriented decomposition is effective way of partitioning problem space of complex system 2. Show key abstractions of agent-oriented mindset are natural means of modelling complex systems

Decomposition: Agents § In terms of entities that have: • own persistent thread of control (active: “say go”) • control over their own destiny (autonomous: “say no”) § Makes engineering of complex systems easier: • natural representation of multiple loci of control § “real systems have no top” (Meyer) • allows competing objectives to be represented and reconciled in context sensitive fashion

Decomposition: Interactions § Agents make decisions about nature & scope of interactions at run time § Makes engineering of complex systems easier: • unexpected interaction is expected § not all interactions need be set at design time • simplified management of control relationships between components § coordination occurs on as-needed basis between continuously active entities

The Match Process 1. Show agent-oriented decomposition is effective way of partitioning problem space of complex system 2. Show key abstractions of agent-oriented mindset are natural means of modelling complex systems

Suitability of Abstractions § Design is about having right models § In software, minimise gap between units of analysis and constructs of solution paradigm • OO techniques natural way of modelling world

The Adequacy Hypothesis Agent-oriented approaches can enhance our ability to model, design and build complex distributed software systems.

The Establishment Hypothesis As well as being suitable for designing and building complex systems, agents will succeed as a software engineering paradigm [NB: will be complementary to existing software models like OO, patterns, components, …]

Agents Consistent with Trends in Software Engineering § Conceptual basis rooted in problem domain • world contains autonomous entities that interact to get things done

Agents Consistent with Trends in Software Engineering § Conceptual basis rooted in problem domain § Increasing localisation and encapsulation • apply to control, as well as state and behaviour

Agents Consistent with Trends in Software Engineering § Conceptual basis rooted in problem domain § Increasing localisation and encapsulation § Greater support for re-use of designs and programs • whole sub-system components (cf. components, patterns) § e. g. agent architectures, system structures • flexible interactions (cf. patterns, architectures) § e. g. contract net protocol, auction protocols

Agents Support System Development by Synthesis An agent is a stable intermediate form • able to operate to achieve its objectives and interact with others in flexible ways construct “system” by bringing agents together and watching overall functionality emerge from their interplay • well suited to developments in: § open systems (e. g. Internet) § e-commerce

Conclusions § Agents are a new model of computation § Basic concepts are: • Agents • Interactions • Organisations § Agents well suited to developing complex distributed applications

Further Reading § M. Luck, P. Mc. Burney and C. Preist (2003) “Agent Technology: Enabling Next Generation Computing (A Roadmap for Agent Based Computing), Agent. Link, 2003. § N. R. Jennings (2000) “On Agent-Based Software Engineering” Artificial Intelligence, 117 (2) 277 -296. § H. S. Nwana (1996) “Software Agents: An Overview” The Knowledge Engineering Review, 11 (3). § M. Luck, R. Ashri, M. d’Inverno (2004) Agent-Based Software Development, Artech House, 2004.

Part II Conceptual Infrastructure: The SMART Agent Framework with Mark d’Inverno, University of Westminster, UK

The SMART View: Objects

Agents

Autonomous Agents

Conceptual Infrastructure Structured, Modular Agent and Relationship Types (SMART) Entities Objects Agents Autonomous Agents §Entities have attributes §Objects are entities with capabilities §Agents goals are objects with §Autonomous agents are agents able to generate their own goals (based on motivations)

Entities § Abstraction with • • Actions Attributes Goals Motivations

Objects, Agents, Autonomous Agents § Objects are entities with non-empty actions § Agents are objects with non-empty goals § Autonomous Agents are agents with non-empty motivations

Motivation § Motivations give rise to goals • Greed • Hunger • Etc § Robbing a bank – greed § The why as opposed to the what

Engagement § § One agent satisfies the goals of another Can build up chains of engagements Autonomous agent at head of chain Cooperation involves two autonomous agents

Relationships § Ownership • When no others engage an agent § Direct ownership • When no chain is involved § Other categories are possible

Understanding Agent Systems

Part III Technical Infrastructure: Agent Implementation through Jini with Ronald Ashri, University of Southampton, UK

Overview § Goals and roadblocks § Required solutions § What is infrastructure support for agentbased systems? § A layered approach to infrastructure § § Intelligent agents, mobile agents and middleware Concept-centred view of agent development § Paradigma: Jini-based agent system § Further Work

Goals § Integration of services § Effective information management § Intelligent Environments at work, home and in the community § Minimised workload § Optimised resource handling § More fun in life!

Example: Ambient Intelligence The game is starting in 5 min. Milk is out of date Home Community There is an urgent mail! Hi, I am a new mobile phone Just let me know how you would like me to take care of things. Autonomous Agent

Roadblocks § A multitude of computing environments (workstations, embedded, mobile) § A multitude of platforms (Windows, Unix/Linux, Macintosh, OS/2, Palm. OS, legacy, …) § Different applications for every task, little integration, too much work expected from the user (direct manipulation). § Complex administration

Required Solutions § § Move from static to dynamic networks Online Communities § § § Dynamic registration of new devices and software Fault-tolerance Move from object-oriented to agent-oriented software engineering § § § Co-ordination amongst software components Delegation of tasks and task sharing Autonomous behaviour

How to provide such solutions § § Middleware technologies to create the appropriate environment Many candidates § § § Service Location Protocol Salutation UPn. P E-Speak Jini All still fairly new but there is progress and reasons to be hopeful

Solutions: Frameworks for agentbased systems § A framework should. . . § § Provide meanings for common concepts Enable comparison and evaluation of alternative designs Provide for subsequent refinement and development Challenge is to provide a framework with a strong conceptual base that can be practically implemented!

Solutions: Design principles § § § A common structure for both practical and theoretical development of agent research Use of existing technologies (Java, XML), especially as it refers to the environment (Jini middleware) Extensibility that will allow for the natural evolution of the framework

Infrastructure support for agentbased systems Basic building blocks required for the development and deployment of an application What are the significant re -usable and domain independent components? PARADIGMA What is an appropriate framework through which to manipulate these components? SMART Aim is support for agent-based systems not general distributed systems. Agent infrastructure should touch upon lower-level issues as well as higher-level.

Heterogeneous Environments LAN Internet LAN Network Community Internet Gatewa y cellular network LAN bluetooth short range wireles s

Research Fields Intelligent Agents • reasoning • negotiation • coalition formation • autonomy • security • communicati on • coordination Middleware • network protocols • reflection • access policies • code • discovery mobility • security • binding resource control • optimization • state migration • security • Mobile Agents

Infrastructure Layers level of abstraction intelligent agents (goal-directed, autonomous operation) mobile agents (resource optimization) middleware (registration, discovery)

Framework Centered View Capabilities, Attributes, Goals, Plans, Motivations Domain specific capabilities Infrastructure support capabilities Agent & Relationship Types (Agent Framework) Agent Execution Environment Middleware Programming Language Support OS + Networking Application Developer Concern Agent Program

Conceptual Infrastructure Structured, Modular Agent and Relationship Types (SMART) Entities Objects Agents Autonomous Agents §Entities have attributes §Objects are entities with capabilities §Agents are objects with goals §Autonomous agents are agents able to generate their own goals (based on motivations)

PARADIGMA/ act. SMART § Infrastructure that provides base agent concepts to ground development of specific systems without starting from scratch (conceptually), but ……. § Infrastructure that avoids forcing a developer to commit to a particular agent architecture § Infrastructure that facilitates clean separation of agent behaviour from agent description, and promotes modularity of agent construction

Technical Infrastructure § XML for agent specification • XML separates agent function from agent description • No specialised tools required and enables the creation of agent libraries • XML could, eventually, be used for agent packaging to allow mobility. A far less costly mechanism for serialisation when compared to the Java mechanism. § Jini as the network environment • Ready Java-based implementation • Source available and well supported through Jini community (www. jini. org) • Based on lookup, discovery, join, entries, leasing and transactions

Overview Formal Agent Framework Agenthood Autonomy Coordination PARADIGMA Java Jini/Javaspaces XML JVM/RMI Network Communications (TCP/IP)

Technical Infrastructure Capabilities, Attributes, Goals, Plans, Motivations Domain specific capabilities Message Manager Ontology Engine Registration XML-based Serialization/Deserialization Goal Selection Plan Selection Resource Access Control Event Listener … Dynamic Linking … Jini + Java + Unix + TCP/IP Attributes Capabilities Goals/ Plans Motivations Neutral Objects Server Agents Autonomous Engagement Agents Types Contract Types

Jini as the network environment 4. print request Executing Environment Printer User Printer Service 3. retrieval Jini Lookup Service 1. registration Executing Environment Printer Interface Description Printer Interface 2. discovery/lookup

Technical Infrastructure: Jini § § Agent discovery through lookup and join Agent description using entries Agent management via leasing Information sharing can be done using Javaspaces (Linda-like tuple-space)

Conceptual Infrastructure: SMART § Some further definitions: § § § These definitions allow for more dynamic behaviour § § Server Agents - Those agents that are not autonomous Neutral Objects - Those objects that are not agents Once a neutral object is assigned a goal it becomes a server agent When a goal is satisfied it reverts to a neutral object This is a possible solution to the issue of agentification (Shoham, 93) Many more issues covered (interactions, planning, cooperation)

Paradigma: Using Neutral Objects Jini Lookup Service Neutral Object Proxy 2. lookup 1. publish Description 5. call capabilities of neutral object 3. retrieve 4. instatiate Autonomous Agent Neutral Object Proxy Server Agent Neutral Object

Paradigma: Different Execution Models § Execution on server’s JVM (Remote sensor interfaces) § Execution on client’s JVM (New capabilities for engaging agent, network monitoring) § Execution on both client and server (Smart proxy) § Allows for efficient use of computing resources

PARADIGMA § XML is used to describe agents (in terms of attributes, capabilities, goals, etc) § Description are parsed to create component collections that are linked via infrastructure support and domain-specific capabilities § Agent constructors carry necessary information for agent infrastructure and domain-specific capability needs § Capabilities can be dynamically loaded § Agent creation and operation are separate § Currently, Jini is used for agent discovery

Paradigma Architecture Creation Attributes Attribute. Collection DTD Interpreter XML Operation Factory sensing Attribute View. Creator Capabilities Capability. Collection DTD XML Interpreter Factory Capability Goals DTD XML Interpreter Factory Infostore Goal. Base Goal. Selection Goal Plans Plan. Base DTD XML Interpreter Plan. Selection Factory Plan. Base action Motivations DTD XML Interpreter Factory Motive

PARADIGMA § § Decoupling agent behaviour and description Helps deal with environmental constraints – behaviours can vary to suit executing environments while behaviours remain the same § § § Flexibility in evaluation - alternative algorithms (behaviours) can be applied to the same descriptions, and be clearly evaluated against each other Provides alternative forms of mobility Enables libraries of agent components (attributes, capabilities, plans, goals)

Conclusions § § § A layered approach maximizes inter-domain transfer It can employ existing technologies, rather than competing with them, engaging user and developer communities By placing the agent framework at the centre we focus on agent issues and better relate them to non-agent issues Paradigma adopts this approach Through Paradigma, SMART is instantiated, evaluated and refined

Further Work § Apply Paradigma to significant real world problems for further evaluation § Develop conceptual infrastructure to support other issues eg. mobility § Dynamic self modification of capabilities – conceptual and technical development § http: //www. ecs. soton. ac. uk/~ra 00 r/paradigma

Part III Applied Infrastructure: Agent Construction for Mobile Devices with Ronald Ashri, University of Southampton, UK

Overview § § § Challenges for agent development on mobile devices Desiderata for an agent construction model Agent Construction Model Example Implementation Conclusions

Challenges § § “Standard” challenges § Heterogeneity § Dynamicity § Diverse Application Domains Mobile devices add § Memory, storage, processing power, operating power limitations § Unstable network connectivity § Different devices for different tasks/trips § Operating through changing organisational domains

Desiderata § Agent construction framework that is able to deal with these challenges through: § § § Architecture neutrality Modularity Powerful reconfiguration (at run-time when possible)

Architecture Neutrality § § Developers need to produce agent-based solutions for a range of application domains Constraining a developer to one specific architecture (e. g. BDI) is not practical Providing a really good generic architecture doesn’t work either Agent construction model must be architecturally neutral

SMART § § Provides the underlying theoretical approach It is already architecturally neutral Has been used to describe a range of agent architectures However, deals only with the description of agent systems not their construction

Descriptive Specification (SMART) § What an agent is and does § § Attributes Capabilities Goals Motivations PDA Storage: 64 MB Connectivity: 802. 11 b Owner: Ronald Ashri --Join Research Group Receive messages from other agents Negotiate meetings Notify User --Arrange a meeting with Mike --Keep meetings as short as possible

Structural Specification § The main building blocks of the agent § § § Sensors Actuators Controllers Infostores Each component is described with attributes (stateless and situation) and capabilities Incoming Message Sensor User Preferences Infostore Message Mailbox Infostore Message Analysis Controller Meetings Negotiation Controller Outgoing Message Actuator User Notification Actuator Agenda Update Actuator

Behavioural Specification § How the agent behaves § § § Links between components Type of messages exchanged Execution sequence of components Incoming Message Sensor User Preferences Infostore Message Mailbox Infostore Message Analysis Controller Meetings Negotiation Controller Outgoing Message Actuator User Notification Actuator Agenda Update Actuator

Reconfiguration § Agent aspects managed through a shell, which directs access to each specification

Example: BDI Architectures § BDI aims to model rational or intentional agency § The symbols representing the world correspond to mental attitudes § Three categories: • informative (knowledge, belief, assumptions) • motivational (desires, motivations, goals) • deliberative (intentions, plans)

BDI Systems § § § BDI = Belief, Desires and Intentions Many agent architectures are BDI based Original system was PRS More recent versions include d. MARS. Other related systems include Agent. Speak(L) and Agentis

Folk Psychology § I believed the tutorial today was at 8 am so I intended to arrive yesterday from London. § I believed the planes were not delayed and desired not to be late so I intended to arrive by 6 pm. § Compelling because • familiar: what it wants, knows and intends - easier to understand predict behaviour. • Other agents can understand predict behaviour • Relationship between these three categories may give us a handle on intelligent action in general.

BDI Architectures § Beliefs - modelling world state. § Desires - choice between possible states. § Intentions - commitment to achieving particular state.

PRS/d. MARS/Agent. Speak(L) § § Beliefs: information about the world Goals: tasks to achieve Plan library: procedural knowledge Intentions: partially instantiated selected plans

Procedural Reasoning System (PRS) Beliefs Sensor input Plan library Action output Interpreter Goals Intentions

BDI Architecture § In general, an agent cannot achieve all its desires. § Must therefore fix upon a subset. § Commit resources to achieving them. § Chosen desires are intentions. § Agents continue to try to achieve intentions until either • believe intention is satisfied, or • believe intention is no longer achievable.

Plans § BDI model is operationalised in PRS/d. MARS agents by plans. § Plans are recipes for courses of action. § Each plan contains: • invocation condition: circumstances for plan consideration; • context: circumstances for successful plan execution; • maintenance condition: must be true while plan is executing, in order for it to succeed; and • body: course of action, consisting of both goals and actions.

Plan Structure Plan Start ? g 1 Invocation Context Body Maintenance Success Failure ? g 2 (otherwise) P 2 P 1 ? g 3 ? g 4 *a 1 P 3 P 4 End 3 !g 1 End 1 !g 2 End 2

Operation 1 § Observe world and agent state, and update event queue to reflect observed events. § Generate new possible goals (tasks), by finding plans whose trigger matches event queue. § Select matching plan for execution (an intended means).

Operation 2 Intention Plan Instance(m) Plan Instance (m-1) § Push the intended means onto the appropriate intention stack in the current set. § Select an intention stack and execute next step of its topmost plan (intended means): • if the step is an action, perform it; • if it is a subgoal, post it on the event queue. Plan Instance(1)

Applications § § Air-traffic control spacecraft systems telecommunications management air-combat modelling

Example: Agent. Speak(L) - Description

Example: Agent. Speak(L) – Structure

Example: Agent. Speak(L) – Behaviour

Implementation - act. SMART § § All concepts implemented in Java, accessible as APIs Implementation of core in J 2 ME Agent. Speak(L) running on Palm m 100 Desktop version configurable via XML

Implementation – act. SMART

Conclusions § § § Development process greatly aided through better understanding of architecture, easier debugging Affords flexibility and allows easy comparison between different approaches (both at a theoretical and implementation level) Can be integrated with existing methodologies or form the basis for one

Further Work § § § Larger scale implementation Library of architectures and architectural patterns Integration with other aspects (relationships, discovery)