Automated Discovery of Emergent Misbehaviour in Agent-Based Models

Automated Discovery of Emergent Misbehaviour in Agent-Based Models Salem Adra and Phil Mc. Minn 15 June 2010 ABM Workshop -Leeds

Outline Background: • Agent-Based modelling Applications Testing Agent-Based Models: • Motivations and Challenges • Automated Discovery of Emergent Misbehaviour: Our Vision • Testing Strategies 15 June 2010 ABM Workshop -Leeds

Background What is Agent-Based computational modelling? • Increasingly popular modelling paradigm • Main Idea: Simulates the interactions of autonomous entities (agents) with each other and their local environment to predict higher level emergent patterns • Bottom-up procedure The focus goes into the individual interacting units • Benefits: Deeper understanding of the interacting agents and their emergent behaviour 15 June 2010 ABM Workshop -Leeds

Background Example of some Agent-Based Modelling Frameworks: FLAME (C Based) MASON (Java Based) Re. Past (Java Based) Swarm (Objective C) 15 June 2010 ABM Workshop -Leeds

Background Agent-Based Models Examples The Epitheliome Project: (Part of the Physiome Project, a worldwide public domain effort to provide a computational framework for understanding human physiology) Goal: Modelling the Emergent Behaviour of Epithelial Cells Benefits: • Better Understanding of Cell Proliferation and self-organisation • Capability to explore the roles of a single parameter in a complex biological system • Helps in the development of reconstruction methods for artificial human skin 15 June 2010 ABM Workshop -Leeds

Background Agent-Based Models Examples Prediction and validation about Epithelial scratch wound healing process 15 June 2010 ABM Workshop -Leeds

Background Agent-Based Models Examples Modelling of cardiac cells and tissues 15 June 2010 ABM Workshop -Leeds

Background Agent-Based Models Examples Macroeconomic Policy Modelling 15 June 2010 ABM Workshop -Leeds

Background Agent-Based Models Examples Social Insects: Pharaohs Ant 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models • Motivation: • ABM are increasingly used to make predictions that can affect organisations and even society • Hidden errors lurking in such models can be very costly • E. g. In the banking sector, losses made by Nat. West, Barclays and Deutsche Morgan Grenfell totalling tens of millions of pounds were blamed on decisions that involved economic model errors • The correctness of such models and their thorough testing is vital • No Formal Testing Technique for ABM has yet been established 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Agents vs. Objects “Object-oriented programmers often fail to see anything novel or new in the idea of agents. … While there are obvious similarities, there also significant differences between agents and objects. ” N. R. Jennings, K. Sycara, and M. Wooldridge, “A roadmap of agent research and development, ” Journal of Autonomous Agents and Multi-Agent Systems, vol. 1, no. 1, pp. 7– 38, 1998. 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Agent vs. Objects: Evolution of Programming Approaches Sources: (1) James J. Odell: "Objects and Agents Compared", in Journal of Object Technology, vol. 1, no. 1, May-June 2002, pp. 41 -53. (2) Parunak, H. Van Dyke, "Go to the Ant": Engineering Principles from Natural Agent Systems, Annals of Operations Research, 75, 1997, pp. 69 -101. 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Agents vs. Objects: Main Differences: AUTONOMY and FLEXIBILITY Some examples of differences between Objects and Agents Objects Agents Behaviour controlled by external entities Self-governed (goals and rules) Always Say “Yes” - “No” is an error Allowed to Say “No” Predictive behaviour (Static Functionality – Unpredictable - Behave differently in which facilitates inheritance) different scenarios (can learn from experience and evolve) 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models • Challenges: • ABM are hard to test • Models often built to predict previously unseen emergent behaviour • Modellers themselves are not sure what to expect • Main purpose for modelling in the 1 st place • Model Complexity • Simulating interactions of thousands and millions of agents is common • Tracing back certain model behaviour to certain agent or event is very hard 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Errors and Artefacts in Agent-Based Modelling Galan et al. (2009), Journal of Artificial Societies and Social Simulation vol. 12, no. 1 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Without visual inspection this bug would have not been caught Most modelling work involve batch processing tasks (i. e. no visualisation) 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Some Suggested Informal Validation Solutions: 1. Application of an ABM to extreme scenarios 2. Re-implementing an ABM using: • • programming paradigms • 15 June 2010 different programming languages, agent-based modelling frameworks ABM Workshop -Leeds

Testing Agent-Based Models Towards Formalising ABM testing The Misbehaviour Project: Project’s Aim: Automated Discovery of Emergent Misbehaviour (ADEM) Develop testing techniques and frameworks for agent-based software that can lead to the detection of systems’ Emergent Misbehaviour 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Towards Formalising ABM testing Approach: Automated Testing Framework composed of 2 main steps Step 1 - Learn and Build up an abstract model of the ABM to be tested Formulate fitness functions (simpler for task oriented ABMs) and constraints that define the model’s normal behaviour) Tools: Daikon (invariant detection), statistical and visual analysis methods Step 2 - Search-Based Testing Process (using a black box approach) Search for scenarios, parameters or settings that might force the ABM to uncover “Misbehaviour” or bring it close to an extreme behaviour (i. e. violating a fitness functions and/or constraint) Tools: Metaheuristic techniques (e. g. GAs, Memetic Algorithms, … etc) 15 June 2010 ABM Workshop -Leeds

Testing Agent-Based Models Start Max Iterations Reached? Yes No Bug Found ADEM Procedure No No Yes Learn Model (DAIKON) Produce new Behaviour (GAs) End 15 June 2010 Misbehaviour Detected? ABM Workshop -Leeds Found Bug

• Case Studies: Cancer Research and BT • 1 - Learn real model (i. e. invariants, agent’s properties and normal behaviour) • 2 - Produce buggy models out of the real models • How? Mutation Testing: Introduction of artificial bugs • 3 – Apply ADEM Procedure: Repeat until Misbehaviour Found or maximum iterations reached IF any Misbehaviour already detected or any Invariant violated Task accomplished ELSE Deploy metaheuristics to search for Misbehaviour END 15 June 2010 ABM Workshop -Leeds

Th an Yo k u. 15 June 2010 Th e En d. ABM Workshop -Leeds