Automated Testing of Classes Alessandro Orso Politecnico di

Automated Testing of Classes Alessandro Orso Politecnico di Milano - Georgia Institute of Technology http: //www. cc. gatech. edu/~orso Ugo Buy University of Illinois at Chicago Mauro Pezzè Politecnico di Milano – Università di Milano Bicocca Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Automated class testing Problems with class testing – Object = data + operations – Behavior of objects highly depends on their state – Encapsulation and Information hiding the state is hidden Breaking of the Generation of encapsulation message sequences Formal specs Scripting Source code The overall goal is the automated, code-based generation of sequences of messages to exercise an object (i. e. , an instance of a class) Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Class and state: The Coin. Box example Class Coin. Box { int total; int cur. Qtrs; int allow. Vend; public: Coin. Box() { total=0; allow. Vend=0; cur. Qtrs=0; } void add. Qtr(); void return. Qtrs(); void vend(); }; void Coin. Box: : return. Qtrs() cur. Qtrs=0; allow. Vend=0; } void Coin. Box: : add. Qtr() { cur. Qtr=cur. Qtr+1; if (cur. Qtr>1) allow. Vend=1; } void Coin. Box: : vend() { if (allow. Vend) { total=total+cur. Qtrs; cur. Qtrs=0; allow. Vend=0; } } Automated Testing of Classes - 08/22/2000 - Alessandro Orso { Fault: method return. Qtrs does not reset variable allow. Vend Possible failure: Coin. Box() add. Qtr() return. Qtr() vend() free drink!

Steps of the technique 1. Data-flow analysis to identify “paths of interest” within the class 2. Symbolic execution to express the semantics of the class in terms of pre- and post-conditions 3. Automated deduction techniques to generate sequences of messages for the class that exercise those paths Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Step I: Data flow analysis Step I Class Coin. Box 1 Coin. Box() 2 total=0 3 allow. Vend=0 10 return. Qtrs() 18 vend() 13 add. Qtr() 14 a t=cur. Qtrs+1 19 if(allow. Vend) 11 cur. Qtrs=0 DEF T 14 b cur. Qtrs=t 20 a t=total+cur. Qtrs USE 12 exit 15 if(cur. Qtrs>1) T 5 exit 20 b total=tmp 20 b total=t 16 allow. Vend=1 4 cur. Qtrs=0 21 cur. Qtrs=0 F 17 exit 22 allow. Vend=0 Virtual Node 23 exit 23 Vend() Automated Testing of Classes - 08/22/2000 - Alessandro Orso F

DU associations Step I DUA# 1 2 3 4 5 6 7 8 9 10 11 12 13 variabile cur. Qtrs allow. Vend total cur. Qtrs allow. Vend cur. Qtrs allow. Vend total cur. Qtrs definition Coin. Box (4) Coin. Box (3) Coin. Box (2) return. Qtrs (11) vend (22) return. Qtrs (11) add. Qtr (14 b) add. Qtr (16) vend (20 b) vend (21) Automated Testing of Classes - 08/22/2000 - Alessandro Orso use add. Qtr (14 a) vend (20 a) vend (19) vend (20 a) add. Qtr (14 a) add. Qtr (15) vend (20 a) vend (19) vend (20 a) add. Qtr (14 a)

Step II: Symbolic execution Step II For each path in each method of the class: – Execution conditions: ( cur. Qtrs>0 ) – Relation between input and output variables: allow. Vend'=1, cur. Qtrs'=cur. Qtrs+1 – Set of defined attributes def={cur. Qtrs, allow. Vend} Limitations: – Approximations required for loops – Restrictions on the program constructs that symbolic execution can handle Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Conditions for the Coin. Box example add. Qtr (cur. Qtrs>0) allow. Vend'=1, cur. Qtrs'=cur. Qtrs+1 def={cur. Qtrs, allow. Vend} (cur. Qtrs==0) cur. Qtrs’=1 def={cur. Qtrs} (cur. Qtrs>0) 0) (cur. Qtrs == Step II add. Qtr() cur. Qtrs=cur. Qtrs+1 if(cur. Qtrs>1) T allow. Vend=1 exit Automated Testing of Classes - 08/22/2000 - Alessandro Orso F allow. Vend'=1, cur. Qtrs'=cur. Qtrs+1 cur. Qtrs'=1

Step III: Automated deduction Step III For each DU association , identified with data flow analysis, build the method sequence: Method that contains the definition of v Method that contains the use of v Def-clear path w. r. t. v Automated Testing of Classes - 08/22/2000 - Alessandro Orso

The method invocation tree Step III • node m = a path in m and the associated pre-condition cond(m) m • successors m’ (of m) = all paths pj m() whose post-conditions do not contradict cond(m) m’ if(a>0) T cond(m’) is F conjunction of the • cond(m), without clauses implied by the if(b<3). . . F post condition of pi T m’ • precondition of pi. . . F exit Automated Testing of Classes - 08/22/2000 - Alessandro Orso (b>=3) && (c > d) (a>0) && (b>=3) m’ (a>0) && (c==d) (c > d) (a’ = 1) (c == d) (b’ = 9) (c < d) (b’ = 2)

Building the tree Step III 1. root = a path in mu that traverses the use 2. If possible, add md to the tree and explore only the sub-tree rooted at md, else start exploring the paths that do not imply the condition of their predecessor 3. Once md has been added, the next goal is to add a constructor • Stop when: – Both md and a constructor have been added: test case – No more nodes can be added: infeasible DUA – Tree depth reaches a user-provided threshold : fail Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Building the deduction tree for DUA #7 Step III use var: cur. Qtrs def: return. Qtr node 11 use: vend node 20 a vend def (Allow. Vend!=0) return. Qtrs (cur. Qtrs>0) add. Qtrs add. Qtr (cur. Qtrs>0) add. Qtr (true) add. Qtr return. Qtrs cur. Qtrs'=0 def={cur. Qtrs} (cur. Qtrs == 0) && (Allow. Vend!=0) (cur. Qtrs>0) allow. Vend'=1 add. Qtr vend def={cur. Qtrs, allow. Vend} (cur. Qtrs >cur. Qtrs'=cur. Qtrs+1 0) && (cur. Qtrs = 0) (cur. Qtrs==0) Coin. Box def={cur. Qtrs} (true) return. Qtrs (true) (Allow. Vend == 0) total'=total constructor cur. Qtrs'=1 cur. Qtrs'=0 Test case: Coin. Box(), add. Qtr(), return. Qtr(), vend() def={cur. Qtrs} Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Characteristics of the technique • Focuses on problems related to the state (instance variables) • Generates test cases from the source code (no specification required) • Uses a stepwise automated approach (partial results can be used) Automated Testing of Classes - 08/22/2000 - Alessandro Orso

The experimental setting C++ Parser C++ Code IL IL Analyzer Data Abstract Graph Flow Analyzer DF Tool Symbolic Executor DUAs + Pre/Post Conditions Automated Deduction Automated Testing of Classes - 08/22/2000 - Alessandro Orso Message Sequences

Summary and future work • Technique for automated generation of test data for classes • Combination of existing techniques in a new context: dataflow analysis, symbolic execution, and automated deduction • Useful intermediate and partial results – DUAs – Pre/Post – Subset of test cases • Results of the preliminary experiments are encouraging • A tool is under development (first prototype due by the end of the year) • Future work – Experimentation – Extension to non-primitive attributes Automated Testing of Classes - 08/22/2000 - Alessandro Orso

Questions? Automated Testing of Classes - 08/22/2000 - Alessandro Orso