Software Testing Methods and Tools Ahmad A Saifan

Software Testing Methods and Tools Ahmad A. Saifan Yarmouk University 1

Outlines • Mutation testing – Web applications – Mobile agent systems – Android applications – Other domains… • Test Case Reduction – Data mining techniques 2

Mutation Testing- Phase 1 Mutant Generation 3

Mutation Testing: Phase 2 Test Execution Test Suite Test cases applied to the mutant program -Killed mutant -Survived mutant 4

5

Testing Web Applications • A web application is a software that can be used through the web, where users can access, using Hypertext Transfer Protocol (HTTP) requests, via a graphical users' interface running in their computers. • Challenges and Limitations in Web Applications : – Web application control flow depends on: • user input: • user location : – The browser interaction when user clicks backward, forward or brows new links within the web page makes it complex to build control flow. – The different technologies that are integrated, within a web page, make it difficult to analyze and model. – These technologies include client-side technologies: (HTML, cascade style sheet), server-side technologies: (PHP, Perl, XML, SQL: Structure Query Language, SSL). 6

Testing Web applications • Graph coverage • Mutation testing • Input space partition – generate a large number of tests – Not effective without strong automation Mahmoud Bani Ata, 2015, master thesis, Investigating the Ability of Testing Web Applications Based on Traditional Software Testing 7 Techniques

Testing Web applications: Graph coverage Web site http: //www. d 3 watuna. com/ -1284 prime paths - 196790 test paths 8 size 817 Number of Nodes 8 In-links 38 Out-Links 38

Testing Web applications Graph coverage How many test paths can be generated from this graph? Web site www. jordantalent. com 9 size 7000 Number of Nodes 48 Inlinks 403 Out. Links 403

Testing Web applications Graph coverage What about this graph? Web site http: //www. roya. tv/ 10 size 25000 Number of Nodes 30 Inlinks 261 Out. Links 261

Testing Web applications Graph coverage • Test paths minimization techniques ? • Automatically generate the prime paths? 11

Testing Web applications: Mutation Testing • Mutation Operators: – SQL data retrieving – data manipulation – Domain name and IP address look up – internet protocol and service information – HTTP – connection to server and to database. 12

Testing Web applications: Mutation Testing Mutation Operator- Examples • AWCS- Add WHERE Clause to select statement Mutant Code Original Code SELECT * FROM ‘department’ ; ∆ SELECT * FROM ‘department’ where ‘deptid’ = 1; • CIPA: Change the IP address parameter of the function gethostbyaddr Original Code Mutant Code $ip = gethostbyaddr(87. 236. 233. 10) ∆ $ip = gethostbyaddr(76. 162. 186. 2) Echo $ip // yuccweb. yu. edu. jo //rev. opentransfer. com. 13

Mutation testing-web applications tool 14

Mutation testing-web applications Results SQL Data Retrieving operators K 2 2 10 3 K K K K K 2 K K 13 10 2 2 Equiv K K killed K K K ANLO 2 K K E E E E K K E E 4 K K ALO K K K E K K 18 K CSF K K K K 19 K K E E E K K K RDK K K RWCS K K AWCS K K CNR K K ADIK K K SAOO K K E E K K ROFW COS Vacancies_ar View vacancies * View vacancies_ar clients Clients_ar positions Edit_advertisment Get_image View candidates changepass cv CV_ar killed ROBK 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 15 Login. php Login_ar Advertisement. php Advertisement_view* Advertisement_view_ar* Advertisement_view_single AOBK 1 2 3 4 5 6 7 8 RLCS PHP Files ALCS Sr. K K K K 5 5 10 10 5 7 1 1 1 1 0 K K K K K K K K 23 5 5 7 7 5 5 5 5 1 1 1 1 0 0 1 2 1 1 135

Mutation testing-web applications Results 16

Mutation testing-web applications Results # of Generated Mutants Killed 188 Equivalent Mutation Score 168 20 89% Web site www. jordantalent. com 17 size 7000 Numb er of Nodes 48 In. Outlinks Links 403

Mobile Agent System (MAS) • MAS is a special kind of distributed system in which the agent software can move from one physical host to another 18

Mutation Testing - MAS • Change Dispatch Destination (CDD). 19

Mutation Testing – MAS Publication • Ahmad. A. Saifan, J. Dingel, J. S. Bradbury and E. Posse, "Implementing and Evaluating a Runtime Conformance Checker for Mobile Agent Systems, " 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation, Berlin, 2011, pp. 269 -278 • Ahmad A. Saifan, . and Wahsheh, H. A. 2012. Mutation operators for JADE mobile agent systems. In Proceedings of the 3 rd international Conference on information and Communication Systems (Irbid, Jordan, April 03 - 05, 2012). ICICS '12. ACM, New York, NY, 1 -5. 20

Mutation testing –Android App 21

µ-Android Tool 22

Mutant Testing – Android App K-9 Mail Operators Universal Music Player Marker GPS Log 23 Total Mutants Killed Mutants Equivalent Mutants Mutation Score RSED CSAMP RGDRV RQMPV RIM RUM RDM MACP Total RGCPRV RIPRV RPM RSTP RCM RDCM RCWDM RICRV Total CCBPV CBDRV CRO CSAP RSAM CSCP RSCM CSSWPV RSCFM CSSTPV RSSM CSTSPV RSTSM CSMPV CUMPV Total SBDM RGAGBM RGSM RGAGLM RGEM CSPPV 135 57 13 12 6 19 8 3 253 6 7 9 5 4 5 6 5 12 59 4 40 4 3 7 7 24 24 4 4 5 5 5 3 149 N/A 8 7 12 N/A 1 133 44 12 11 6 19 8 3 236 6 7 9 5 4 5 6 5 12 59 4 40 4 3 6 7 21 24 3 3 4 5 5 3 141 8 7 12 1 2 13 1 1 0 0 17 0 0 0 0 1 0 3 0 1 1 1 0 0 0 8 0 0 98. 52% 77. 19% 92. 31% 91. 67% 100. 00% 93. 28% 100. 00% 100. 00% 85. 71% 100. 00% 87. 50% 100. 00% 75. 00% 80. 00% 100. 00% 94. 63% 100. 00% • • • Replace Standard Extra Data (RSED) Correct. Code send. Intent. put. Extra(Intent. EXTRA_TEXT, "Text"); Mutant Code send. Intent. put. Extra(Inten t. EXTRA_STREAM, "Text. ");

Mutant - Results 24

Publication • Ahmad Saifan, Ahmad alzyoud, Mutation Testing to Evaluate Android Applications, Information and Software Technology. [ISI], submitted, Dec 5 th 2016. 25

Task 26

Test Case Reduction • Data Mining – Clustering – Classification 27

Test Case Reduction using Data Mining Technique • Ahmad A. Saifan, Emad Alsukhni, Hanadi Alawneh, Ayat Al-Sbaih, (2016). Test Case Reduction using Data Mining Technique, International Journal of Software Innovation, (IJSI). Volume 4 Issue 4, October 2016 Pages 56 -70. • jhh 28

Sample of Data Set used Test Case # instructions Branches Lines Methods 1 3. 0 87. 0 50. 0 12. 0 1. 0 2 9. 0 76. 0 28. 0 1. 0 3 6. 0 89. 0 50. 0 40. 0 1. 0 4 15. 0 83. 0 50. 0 97. 0 1. 0 5 3. 0 94. 0 19. 0 1. 0 6 5. 0 81. 0 50. 0 31. 0 7 1. 0 100. 0 5. 0 1. 0 8 1. 0 100. 0 2. 0 1. 0 9 1. 0 100. 0 8. 0 10 1. 0 100. 0 8. 0 11 1. 0 100. 0 7. 0 12 1. 0 100. 0 7. 0 13 1. 0 100. 0 6. 0 14 1. 0 100. 0 8. 0 15 2. 0 92. 0 50. 0 11. 0 16 5. 0 98. 0 50. 0 15. 0 17 5. 0 98. 0 50. 0 15. 0 18 5. 0 98. 0 50. 0 15. 0 19 1. 0 100. 0 11. 0 20 1. 0 100. 0 5. 0 1. 0 21 1. 0 100. 0 5. 0 1. 0 22 1. 0 100. 0 5. 0 1. 0 23 1. 0 100. 0 6. 0 1. 0 24 1. 0 100. 0 6. 0 1. 0 25 29 Complexity 1. 0 100. 0 5. 0 1. 0

Results 504 test cases 93 test cases Test Case Visualization Before Reduction Code Coverage Instructions Branches Lines Methods Test Case Visualization After Reduction Before Reduction 88. 2% 76. 7% 83. 4% 98. 1% After Reduction 87. 5% 81. 1% 78. 2% 98. 1% The coverage Before and After Reduction 30

Test Case Reduction using Data Mining Classifier Techniques • Ahmad A. Saifan, Test Case Reduction using Data Mining Classifier Techniques. (2016). Journal of Software. Volume 11, Number 7. PP 656 -663. 1) Generate test cases from the system under test – 493 test cases were generated 31

Test Case Reduction using Data Mining Classifier Techniques • 2) Data Set Attributes 32

Sample of data set TC_ID 1 2 3 TC_Class. Coverage 5 5 5 TC_Method. Coverage 8 8 8 TC_Input 2, 4, ” 234” 1, 4, ”a 23” 2, 4, ”a 2 c” Class Label Irredundant Redundant • Cover class number 5 • Cover method number 8 • Has 3 inputs 2, 4, ” 234” • An irredundant class 33 3/19/2018

Test Case Reduction Rules Is a set of test cases, where Defined to cover one of the following: A test cases ti is considered redundant with tj if it is satisfied with all of the three rules 34 3/19/2018

Test Case Reduction using Data Mining Classifier Techniques 3) Analyse Results Preprocessing: removing empty instances and the attributes that have less information gain value 35

Ideas • The quality attributes of Android apps – Android Software metrics – tools 36