Automated assume-guarantee reasoning for component verification Dimitra Giannakopoulou

Automated assume-guarantee reasoning for component verification Dimitra Giannakopoulou (RIACS), Corina Păsăreanu (Kestrel) Automated Software Engineering NASA Ames Research Center Moffett Field, CA, USA

System-Level Verification for Autonomous Systems Verification is essential for autonomy insertion in missions executive plans commands Objectives Detect integration problems early Support module-based verification (for scalability) Design for verification Approach Software architecture design and verification Module verification with generated assumptions Design patterns / architectures for autonomous systems

Compositional Verification M 1 satisfies P? A M 2 Decompose properties of system (M 1 || M 2) in properties of its components Does M 1 satisfy P? typically a component is designed to satisfy its requirements in specific contexts / environments Assume-guarantee reasoning: introduces assumption A representing M 1’s “context” Simplest assume-guarantee rule 1. A 2. true M 1 M 2 P A 3. true M 1 || M 2 P “discharge” the assumption

Approach Infer assumptions automatically Two novel solutions developed 1. Algorithmic generation of assumption (controller); knowledge of environment is not required 2. Incremental assumption computation based on counterexamples, learning and knowledge of environment

Applications Verification produces more precise answers true – property satisfied in all environments false – property falsified in all environments assumption A – property true in environments where A is true Support for compositional verification Property decomposition Assumptions for assume-guarantee reasoning Runtime monitoring of environment assumption monitors actual environment during deployment may trigger recovery actions Component retrieval, sub-module construction, …

Implementation in LTSA Components modeled as labelled transition systems (LTS) an LTS M has a communicating alphabet M internal actions represented by action “ τ ” LTSs assembled with parallel composition operator “||” – synchronizes shared actions, interleaves remaining actions A property P is a safety LTS (an LTS with no error states) P describes all legal behaviors with respect to P Perr – determinize & complete P; bad behaviors lead to “error” component M satisfies P iff error state unreachable in (M || Perr) Assume-guarantee reasoning assumptions and guarantees are safety LTSs A M P holds iff error state unreachable in (A || M || Perr)

Example Input in Ordererr send in ack || out Output send out ack in

Solution 1 in collaboration with Prof. Howard Barringer, Univ. of Manchester, UK Giannakopoulou, D. , Păsăreanu, C. , and Barringer, H. , “Assumption Generation for Software Component Verification”, in Proc. of the 17 th IEEE International Conference on Automated Software Engineering (ASE 2002). Awards: Best paper, ACM Distinguished Paper Award

The Problem Given component C, property P, and the interface of C with its environment, generate the weakest environment assumption A such that: assuming A, C ╞ P Weakest means that for all environments E: (E || C ╞ P) IFF E╞ A

Assumption Generation Step 1: composition, hiding, minimization Property true! (all environments) Step 2: backward propagation of error with Property false! (all environments) Step 3: property extraction (subset construction and completion) Assumption

The problem 1. A 2. true M 1 M 2 P A 3. true M 1 || M 2 P how are assumptions obtained? developer encodes them explicitly abstractions of other modules our goal: automate assume guarantee reasoning by generating appropriate assumptions for the rules

Framework, iteration i counterexample – strengthen assumption Model Checking Learning Ai 1. Ai M 1 P true 2. true M 2 Ai false true P holds in M 1||M 2 false N real counterexample – weaken assumption error? Y P violated in M 1||M 2

Learning with L* L* algorithm by Angluin, improved by Rivest & Schapire learning based on queries and counterexamples learns an unknown regular language (U over alphabet ) and produces a DFA C such that L (C) = U constructs a sequence of DFAs C 1, C 2, … converging to C needs a Teacher to answer two types of questions: membership queries: is string s in U ? conjectures: for a candidate DFA Ci, is L(Ci) = U ? – answers are (true) or (false + counterexample)

The L* Algorithm General method: maintains a table that records whether strings in belong to U makes membership queries to update it decides to make a conjecture – uses table to build a candidate Ci if Teacher replies true, done! if Teacher replies false, uses counterexample to update the table Characteristics: terminates with minimal automaton C for unknown language U each candidate Ci is smallest any DFA consistent with table has at least as many states as Ci |C 1| < | C 2| < … < |C| produces at most n candidates, where n = |C| # queries: (kn 2 + n logm) m is size of largest counterexample, k is size of alphabet

Learning Assumptions satisfies P? M 1 M 2 1. A 2. true M 1 M 2 P A 3. true M 1 || M 2 P L* learns the weakest assumption Aw for M 1 for all environments E: ( M 1|| E satisfies P IFF E satisfies Aw ) a string s is in L (Aw) iff in the context of s, M 1 satisfies P = Aw = ( M 1 P) M 2 conjectures are intermediate assumptions Ai framework may terminate before L* computes Aw Aw is not available – how do we implement the Teacher ? use model checking

The Teacher Membership query: trace s Simulate s on M 1 || Perr – If it leads to error state, reply false ( s L(Aw) ) – Otherwise, reply true ( s L(Aw) ) Conjecture: Ai Oracle 1: performs (Step 1) Ai M 1 P – If false (with counterexample c): provide c to L* (assumption too weak) – If true: forward Ai to Oracle 2: performs (Step 2) true M 2 Ai – If true: done! (P holds on M 1 || M 2) – If false (with counterexample c): forward c to analysis Counterexample analysis Simulate c on M 1 || Perr – If it leads to the error state: done! (P does not hold on M 1 || M 2) – Otherwise, provide c to L* (assumption is too strong)

Example Ordererr Input Output in send out out in ack We check: true Input || Output Order M 1 = Input, M 2 = Output, P = Order Assumption’s alphabet: {send, out, ack} ack

Membership Queries Input Ordererr Output in send out in ack E Table T S S out ack out send out, ack out, send S = set of prefixes E = set of suffixes out ? Yes! L(Aw)? Simulate on Input || Ordererr

Membership Queries Ordererr Input Output in send out in ack E Table T S S out ack out send out, ack out, send S = set of prefixes E = set of suffixes out Yes! true L(Aw)? Simulate on Input || Ordererr

Membership Queries Ordererr Input Output in send out in ack E Table T S ? No! true out S out ack out send out, ack out, send S = set of prefixes E = set of suffixes L(Aw)? Simulate on Input || Ordererr

Membership Queries Ordererr Input Output in send out in ack E Table T S S out true out ack out send out, ack out, send false No! S = set of prefixes E = set of suffixes L(Aw)? Simulate on Input || Ordererr

Membership Queries Ordererr Input send in send out ack Table T S true out ack out send out, ack out, send false true S = set of prefixes E = set of suffixes out in ack E S Output in false

Candidate Construction Ordererr Input Output in send out out in ack E Table T S S true out ack out send out, ack out, send false true S = set of prefixes E = set of suffixes false 2 states – error state not added to assumption A 1 ack send

Conjectures Ordererr Input Output in send in out in ack A 1: ack Oracle 1: A 1 Input Order ack send Queries out send A 2: send ack out, send Counterexample: c = in, send, ack, in Oracle 1: A 2 Input Order True Return to L*: c = send, ack Oracle 2: true Output A 2 True property Order holds on Input || Output

Another Example Ordererr Input Output in send in out in ack not a real error! Return c to L* out send out Oracle 2: true Output’ A 2 send ack Counterexample: c= send, out real error? Simulate c = send, out on M 1||Ordererr ack A 4: send ack property Order holds on Input || Output’ out send ack, out, send

Ames K 9 Rover Executive Executes flexible plans for autonomy branching on state / temporal conditions Multi-threaded system communication through shared variables synchronization through mutexes and condition variables Systematic translation of design documents into input language of LTSA Several synchronization issues property P: If the Executive threads the value of variable saved. Wakeup. Struct, the Exec. Cond. Checker thread should not read this value unless the Executive clears it first.

K 9 Rover results We check: true Exec. Cond. Checker || Executive P M 1 = Exec. Cond. Checker, M 2 = Executive iteration |Ai| (#states, #trans) result 1 – Oracle 1 1 (5, 24) too weak 2 – Oracle 1 2 (268, 1408) too weak 3 – Oracle 1 3 (235, 1209) too weak 4 – Oracle 1 Oracle 2 5 (464, 2500) (32, 197) not too weak false 8. 639 secs Check M 1 || M 2 || P: (3630 sts, 34653 trans), 0. 535 secs

Related Work Assume-guarantee frameworks Jones 83; Pnueli 84; Clarke, Long& Mc. Millan 89; Grumberg & Long 91; Xu, de Roever & He 97; Henzinger, Qadeer & Rajamani 98; … tool support: MOCHA; Calvin (static checking of Java); … Assumption generation Giannakopoulou, Păsăreanu & Barringer, 2002 Interfaces for Compositional Reachability Analysis Cheung & Kramer 1996, Krimm & Mounier 1997 Learning system models Groce, Peled, and Yannakakis, 2002 Ammons, Bodik, and Larus, 2002

Conclusions & Extensions framework for incremental assume-guarantee reasoning general – relies on standard features of model checkers extend framework to more than two components carry out more & larger case studies alternative approaches for generating assumptions combine with abstraction to deal with source code extend to liveness / fairness properties L Algorithm, Maler and Pnueli, 1995 timed systems

part 2: work in progress

Symmetric Rules assumptions for both components at the same time early termination smaller assumptions take simplest symmetric rule 1. A 1 M 1 P 2. A 2 M 2 P 3. true M 2 A 1 OR true M 1 A 2 true M 1 || M 2 P the rule is sound and completeness is essential for automation

Framework weaken strengthen false learning weaken learning strengthen A 2 A 1 M 1 P A 2 M 2 P false true OR true M 2 A 1 true M 1 A 2 true P holds in M 1||M 2 false counterexample analysis P violated in M 1||M 2

Incorporation of other circular rules a variety of assume guarantee rules exist in the literature usually, they are sound but not complete we make them complete by adding premise 3 to the rule example: A 1 M 1 P A 2 M 2 P co. A 1 || co. A 2 = true M 1 || M 2 P the rule is sound but not complete

Framework weaken learning strengthen false weaken learning strengthen A 2 A 1 M 1 P A 2 M 2 P true co. A 1 || co. A 2 = false counterexample analysis false true P holds in M 1||M 2 P violated in M 1||M 2 false true M 2 A 1 OR true M 1 A 2 false counterexample analysis P holds in M 1||M 2 P violated in M 1||M 2

Incorporation of other circular rules a variety of assume guarantee rules exist in the literature usually, they are sound but not complete we make them complete by adding premise 3 to the rule example – Chandy and Misra 1981: A 1 M 1 P A 2 M 2 P M 1 @ A 1, P , M 2 @ A 2 , P P |= A 1, P |= A 2 true M 1 || M 2 P the rule is sound but not complete

Compositional verification for C check C 1||C 2 |= P use as much information as you can from previous iterations C 1 refine C 2 predicate abstraction M 1 refine M 2 learning framework true C 1||C 2 |= P false spurious counterexample analysis spurious C 1||C 2|=P

Compositional verification for C (details) weaken A 1 learning strengthen A 1 M 1 P A 2 M 2 P true M 1 strengthen A 2 A 1 false predicate abstraction learning weaken A 2 false true M 2 A 1 OR true M 1 A 2 M 2 C 1||C 2|=P false C 1 strengthen M 1 counterexample analysis predicate abstraction C 2 strengthen M 2 C 1||C 2|=P