Software Testing Making sure that what you think

Software Testing “Making sure that what you think that it does and what it does are one and the same. ” Refs: Mc. Connell Chapter 22 Pressman Chapters 16, 17, 22 Kernighan and Pike Chapter 6 CS 3500 UT - 1

Quality l l Quality is fundamental and pervasive across the entire product development process It is achieved by many different means Testing is an important component that requires a strategic approach Testing affirms quality! Software Engineering Methods Formal Technical Reviews Measurement Quality Standards and Procedures CS 3500 Software Configuration Management and Software Quality Assurance Testing UT - 2

Why Software Testing? l l l We aren’t good enough programmers We don’t concentrate enough to keep from making mistakes We don’t use proper design techniques We can’t distinguish what another programmer or customer says and what they really mean We feel guilty that someone else has to test our code We are failures CS 3500 UT - 3

Why Software Testing? l l l We aren’t good enough programmers We don’t concentrate enough to keep from making mistakes We don’t use proper design techniques We can’t distinguish what another programmer or customer says and what they really mean We feel guilty that someone else has to test our code We are failures CS 3500 UT - 4

Testing l l l l Testing is the process of executing a program with the intent of finding an error - as much as 40% of SE effort A good test case is one that has a high probability of finding an as yet undiscovered error A successful test is one that finds undiscovered errors A successful test is one in which no errors are found Testing should find the most errors in the shortest amount of time with the least amount of effort A secondary benefit is that testing can demonstrate to some degree that software appears to be working at an acceptable level of performance However - testing cannot show the absence of defects, it can only show that defects are present CS 3500 UT - 5

Black Box Testing – Given that you know what it is supposed to do, you design tests that make it do what you think that it should do – From the outside, you are testing its functionality against the specs – For software this is testing the interface » What is input to the system » What you can do from the outside to change the system » What is output from the system – Tests the functionality of the system by observing its external behavior – No knowledge of how it goes about meeting the goals CS 3500 UT - 6

White Box Testing – Given knowledge of the internal workings, you thoroughly test what is happening on the inside – Close examination of procedural level of detail – Logical paths through code are tested » Conditionals » Loops » Branches (test both branches) – Status is examined in terms of expected values – Impossible to thoroughly exercise all paths » Exhaustive testing grows without bound – Can be practical if a limited number of “important” paths are evaluated – Can be practical to examine and test important data structures CS 3500 UT - 7

White Box Testing - 2 l The nature of typical errors makes white box testing important – Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed – Our perception about a path not being executed is often incorrect – Typo errors are totally random and could appear in an obscure path “Bugs lurk in corners and congregate at boundaries” l CS 3500 A good approach is to combine the black box and white box to get gray box testing UT - 8

Unit Testing a unit of code in isolation l A Unit is the smallest piece of code that can be tested in isolation l – Smallest piece of code is typically a class or a function – Isolation – separate from all other parts of the system Idea is to test one unit and not multiple units l Usually done by the developer l Involves gray box testing l – That is test the interfaces and examine results (black box) – Make calls to exercise the various internals (white box) CS 3500 UT - 9

Unit Testing Caveats l Testing should be automated – If it’s too hard to do, you won’t do it l Testing should serve a diagnostic purpose – Use multiple test cases with individual pass/fail l Save your test cases – Use them for regression testing l Consider writing test cases first! – Forces you to think about what you’ll be implementing l There are tools to help you – Run multiple test cases, tell you which ones you failed – Show you which portions of code have been “exercised” l Use assertions in programming – Assert. Equals(…, …); CS 3500 UT - 10