Verification of Evolving Software Natasha Sharygina Joint work

Verification of Evolving Software Natasha Sharygina Joint work with Sagar Chaki and Nishant Sinha Carnegie Mellon University

Motivation • Component-based Software – Software modules shipped by separate developers – Undergo several updates/bug-fixes during their lifecycle • Component assembly verification – Necessary on update of any component – High verification costs of global properties

Contribution Assembly A P ? Component C C’ Component

Contribution • Automated substitutability check – Preserving all old behaviors – Allowing new behaviors • Focus on components that have been modified • Allow multiple upgrades simultaneously

Substitutability Check • Given old and new components Ci, C’i and assembly A – For each i check if C’i is substitutable for Ci in A • Two phases: – Containment check • Ensures that behaviors of the old component are contained in its substitutable counterpart – Compatibility check • Safety with respect to other components in assembly: all global specifications are satisfied • Approach: – Obtain a finite behavioral model of all components by abstraction: Labeled Kripke Structures (LKS) – Use model checkers to solve both phases automatically

Abstraction into LKS p • Labeled Kripke Structures – • State-event traces – • Composition semantics – Synchronize on shared actions q !p • Two types of abstraction: – Predicate abstraction (over-approximations: existential transitions) – Modified predicate abstraction (under-approximations: universal transitions) !q

Abstraction into LKS • Program Statements , States • Control flow , Transitions • Transitions depict observable behavior • Automated

Predicate Abstraction into LKS L 1 void OSSem. Pend (…) { L 1: lock = 1; if (x < y) { L 2: lock = 0; … } if (x >= y) { … L 3: lock = 0; … } else { … } } lock=1 x= y) L 3 lock = 0

Verification Specification Model L 1 lock if (P) unlock ERROR L 2 unlock if (!P) L 3 unlock

Verification Specification Model L 1 lock if (*) unlock ERROR L 2 unlock if (*) L 3 unlock

Com. Fo. RT: Component Formal Verification Framework Abstraction Components Assembly Improved Abstraction Guidance Abstraction Refinement Model Verification Counter Example Guided Abstraction Refinement No Spurious Counterexample Yes System OK No Counterexample Valid? Yes

Containment Step 1. Construct LKSs M’ and M such that (C 1) C µ M and (C 2) M’ µ C’ Step 2. Verify if M µ M’ • If yes then from (C 1) and (C 2) C µ C’ and we terminate • Else CE is produced Step 3. Validate CE 1. Check if CE C. If yes proceed to next step. Otherwise refine M and repeat from Step 2. 2. Check if CE C’. If yes than CE C C’, Feedback CE and stop. Else refine M’ (by adding the valid CE) and repeat from Step 2.

Containment (contd. ) • In contrast to Refinement-based approaches – Containment ensures that all behaviors of M are present in M’ – However, containment allows new behaviors in M’ which could be possibly erroneous • Containment check is not sufficient since new components – may not be safe with respect to other components – must satisfy the global behavioral specifications

Learning Regular languages: L* • Forms the basis of the compatibility check • L* proposed by D. Angluin • Polynomial in the number of states and length of counterexample Yes/No a b Is. Member trace ) ( L* learner Is. Candidate DFA D ) ( Model checker a b Minimum DFA Minimally adequate Teacher ±Counterexample/ Yes Unknown Regular Language

Compatibility check • Recursive Assume-guarantee to verify assembly properties R 1: R 2: M 1 || A ² P M 2 || … || Mn ² A M 1 || … || Mn ² P • Generate a (smaller) environment assumption A – A: most general environment so that P holds – Constructed iteratively using L* and R 1, R 2 – Recursion for the R 2 rule

Compatibility check -CE for Ai R 1 : L* Assumption Generation Ai M 1 || Ai ² P true R 2: M 2 || … || Mn ² Ai CE true CE Analysis False CE +CE for Ai True CE

Compatibility check (contd. ) • Generate a most general assumption for each M’ – M 1 = M’ – M 2 = Mn M (all other component LKSs) • Membership queries: – M’ || CE µ P • Candidate queries: – M’ || A µ P – M 2 µ A

Compatibility check (contd. ) • CE analysis: M’ || CE µ P – Yes ) False CE – No ) True CE • Compatibility check infers – Either M’ is substitutable – Or counterexample CE • CE may be spurious wrt. C’ due to the predicate abstraction – CE is present in component LKS M’ – Must refine M’ – Repeat substitutability check

Compatibility check (contd. ) • Dynamic compatibility check – Restarts learning using information from a previously learned candidate – Account for incremental changes between successive assumptions The dynamic nature is critical for 1) checking evolving software; 2) during abstraction refinement where a single component is updated at a time.

Feedback to Developers Feedback - collection of CEs C identified during containment check Feedback Delivery: Given a trace Feedback, and Rep( ) – representation of in terms of program control locations, predicate valuations and actions, the levels of feedback are 1) Rep(Prefix( )) – to denote the divergence point of 2) Rep(Suffix( )) – to show what exact behavior (exact code) of C is missing in C’ 3) Rep’(Pref( )) – to denote location in Ci where changes are to be made

Experiments • Prototype implementation in Com. Fo. RT framework • ABB IPC Module – Deployed by a world leader in robotics engineering systems – 15000+ LOC – modified Write. Message. To. Queue component – checked for substitutability inside the assembly of four components (read, write, queue, critical section) – Over 30 billion states after predicate abstraction • Discovered synchronization bug – Process can incorrectly block while writing to a queue

Related Work • Learning Assumptions: Cobleigh. et. al. – Do not consider state labeling of abstract models – Do not incorporate a CEGAR framework for AG • Compatibility of Component Upgrades: Ernst et. al. – Do not consider temporal sequence of actions in generating invariants • Interface Automata: Henzinger et. al. – Do not have CEGAR, AG

Com. Fo. RT framework: www. cs. cmu. edu/~natalie/

Questions?