Com 1040 Systems Design and Testing Part II

Com 1040 Systems Design and Testing Part II – Testing (Based on A. J. Cowling’s lecture notes) LN-Test 1+2: Introduction to Testing Marian Gheorghe Com 1040 - Testing ©University of Sheffield

Testing -Summary • • Introduction to testing Testing types Unit testing Specification-based testing AJC’s lecture notes at https: //www. dcs. shef. ac. uk/~ajc/campus_only/teach/com 1030/notes/index. html#testnotes MG’s slides at: http: //www. dcs. shef. ac. uk/~marian/teaching/com 1040. html G. J. Myers, The Art of Software Testing, 1979 M. Roper, Software Testing, 1994

The Testing Problem - Triangle Example The aim of this program is to classify triangles. The program accepts three positive whole numbers as lengths of the sides of a triangle. The program classifies the triangle into one of the following groups: • Equilateral: all the sides have equal lengths (return 1) • Isosceles: two sides have equal length, but not all three (return 2) • Scalene: all the lengths are unequal (return 3) • Impossible: the three lengths cannot be used to form a triangle, or form only a flat line (return 4) (it appears in Myers’ book) Find test cases which adequately runs or abruptly breaks it down. An example…

Java Solution int triangle(int a, int b, int c) { int mx, x, y; mx = a; x = b; y = c; if (mx < b) {x = mx; mx = b; } if (mx < c) {y = mx; mx = c; } if (mx >= x + y) {return 4; // impossible} if (a == b && b == c) {return 1; // equilateral} if (a == b || b == c || a == c) {return 2; // isosceles} return 3; // scalene }

Java Solution & Test int triangle(int a, int b, int c) { int mx, x, y; mx = a; x = b; y = c; if (mx < b) {x = mx; mx = b; } if (mx < c) {y = mx; mx = c; } if (mx >= x + y) {return 4; // impossible} if (a == b && b == c) {return 1; // equilateral} if (a == b || b == c || a == c) {return 2; // isosceles} return 3; // scalene } White box: code coverage (3, 5, 3)

Java Solution & Test int triangle(int a, int b, int c) { int mx, x, y; mx = a; x = b; y = c; if (mx < b) {x = mx; mx = b; } if (mx < c) {y = mx; mx = c; } if (mx >= x + y) {return 4; // impossible} if (a == b && b == c) {return 1; // equilateral} if (a == b || b == c || a == c) {return 2; // isosceles} return 3; // scalene } White box: code coverage (3, 5, 3) Correct!

Motivation for Software Testing • Testing part of any scientific and engineering activity • Software testing needed – errors are unavoidable: psychology of testing - limited capacity and performances of the human memory layers • short term • medium term • long term – history of software • Ariane 5 failure, 1996: http: //www. ima. umn. edu/~arnold/disasters/ariane 5 rep. html • Genomic data, Science 2007

Errors in Software Processes Errors may occur in any stage of a software project: • requirements capture: misinterpreted or incomplete aspects, incorrect statements etc • specifications: wrong/inadequate notations, missing reqs, unnecessary components etc • design: wrongly reflected specifications, wrong diagrams etc • code: syntax, logical, I/O errors, format etc

Definitions of Testing • Hetzel: Any activity aimed at evaluating an attribute or capability of a program or system. • Myers: The process of executing a program with the intent of finding errors. • IEEE: The process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements or to identify differences between expected and actual results.

Testing Processes • Test case • Test set • Test script • Test execution • Test analysis • Test suite Anatomy of a test case: • What are the parts of a test case? • a description of input condition(s) • a description of expected results • Where do ‘‘expected results’’ come from?

Brief History of Software Testing* • Debugging, up to 1956 • Demonstration, 1957 -1978 • Destruction, 1979 -1982 • Evaluation, 1983 -… • Focus on shorter development cycles, after 1990 *Some details and examples follow from http: //www. cise. ufl. edu/class/cen 6070/lecture_notes. html - University of Florida

Performing Testing When might testing guarantee an error-free program? • • • When branch, condition, and loop coverage are achieved When dataflow testing is utilized When path and compound condition coverage are achieved When all combinations of all possible input and state variable values are covered (None of the above ? )

Performing Testing When Might Testing Guarantee an Error-Free Program? • • • When branch, condition, and loop coverage are achieved When dataflow testing is utilized When path and compound condition coverage are achieved When all combinations of all possible input and state variable values are covered = exhaustive testing (None of the above)

Exhaustive testing Example: A module has 2 input parameters. Word size is 32 bits. Testing is completely automated: 100 nanoseconds are required for each test case. Question: How long would it take to test this module exhaustively, i. e. , covering every possible combination of input values?

Exhaustive Testing - Answers • Short Answer: too long… • Long Answer: 64 -9 2 X 100 X 10 > 57, 000 years! 3600 X 24 X 365

Properties of Testing Methods • Construction of a test set of reasonable size • Any bigger test set should show better achievements • Relevant aspects of the product are covered • Catch adequate faults

Testing types Various classifications • Random testing – generate random test sets • Implementation-based testing (white box, glass box – start from code) • Specification-based testing (black box – use the, formal or informal, specification) • Hybrid testing – mixture of the above Other classifications consider the amount of code (unit vs integration, system testing), performance (stress testing), usability (acceptance testing) etc. We focus on (1) white box unit testing and black box integration testing utilising category-partition method

White box unit testing You have learned about JUnit (Com 1020 – 2 nd lecture) public class My. Class { public static double my. Meth(Params) { … some code } … } Current class, method Testing suite public class Test. My. Class { @Test public void f. Test. Some. Functionality() { assert. Equals(val, My. Class. my. Meth(some. Vals), err. App); } … }

JUnit testing principle Through assert. Equals(val, My. Class. my. Meth(some. Vals), err. App); results returned by a method, my. Meth(), of a class, My. Class, (in a certain stage of its development) are compared with predefined values, val, for some given inputs, some. Vals. • • Reminds pre- and post-conditions utilised by formal methods, including Z, to specify correct behaviour Problem: how do we choose val and some. Vals

Aim of unit testing for Crossover • • Test every unit of code – java method: • As complete as possible code coverage • Using category-partition to identify suitable inputs – remember (3, 5, 3) for the “triangle” problem Generate suitable test sets

Specification-based methods • • • Starting from specifications May apply to unit as well as integration and system testing Specifications may be informal – plain English or (semi)formal – diagrams or specific notations Z may be suitable for black box unit testing X-machines for black box system testing: appropriate sequence of correct functions Two more concepts, related to data set selection, apply: equivalence partition and boundary conditions

Equivalence partition: Data set identification • Exhaustive testing is not possible • Identify in the set of input data “equivalent” elements – leading to similar behaviour; ex in triangle program (3, 5, 3), (30, 50, 30), (4, 6, 4) show the same behaviour and result, whereas (3, 7, 3) doesn’t! why? • Enough to use one representative per equivalent class

Example Mail-order firm wants to use a computer system to handle customer accounts. In particular, at the end of each month, they require the system to print out a statement for each customer, detailing their purchases during the month together with their current credit rating. The system must also send out invoices to customers with outstanding payments, and should not send anything to customers who bought nothing in the past month, and owe nothing for earlier items. • We need to partition: customers set, their balances, orders set, addresses set

Example cont Mail-order firm wants to use a computer system to handle customer accounts. In particular, at the end of each month, they require the system to print out a statement for each customer, detailing their purchases during the month together with their current credit rating. The system must also send out invoices to customers with outstanding payments, and should not send anything to customers who bought nothing in the past month, and owe nothing for earlier items. • Customer partition: no customers, one customer, more than one - 3 • Balance partition: customers with negative, positive, exact – 3 • Orders partition: no orders, one page, more than one – 3 • Addresses partition: 1 st class, 2 nd class, Eu, rest of the world - 4

Example - analysis Mail-order firm wants to use a computer system to handle customer accounts. In particular, at the end of each month, they require the system to print out a statement for each customer, detailing their purchases during the month together with their current credit rating. The system must also send out invoices to customers with outstanding payments, and should not send anything to customers who bought nothing in the past month, and owe nothing for earlier items. • If independent cases then 3 3 3 4 equivalence classes, but • Empty set of customers leads to one test • No need to consider 3 3 4 cases for one and many customers • So, 3 3 4 + 1 would suffice

Conclusions • Concepts related to testing introduced • Need of testing showed • Testing capabilities discussed • White and black box testing defined and white box unit testing illustrated