Testing Agile OO Robert Johnson Melissa Schwartz

Testing: Agile & OO Robert Johnson Melissa Schwartz

Testing Object-Oriented Software Systems Harry M. Sneed

Object Oriented Systems: The advantage is a disadvantage in terms of testing: a. flexibility and usability o many ways to use and therefore many needed tests "One can summarize that the greater the variability and flexibility of a piece of software, the more the usage possibilities are, then the more there is to test. "

The solution to the challenge? a. Automation

Higher cost: "Boris Beizer, the dean of Software testing, wrote in an article in the American Programmer, saying that it would cost a lot more to test 00 software than to test conventional procedural software – perhaps 3 or 4 times as much. "

Why? a. object-oriented class hierarchy has to be retested for every context in which it is reused • little differences in the environment can cause big side effects "James Martin, the guru of data – driven software development pointed out, that reusing classes from class libraries was like eating spaghetti. If you wanted to pull out one strip all of the other strips come along too. They are all intertwined with one another. So it is not possible to select and test individual classes. If you want to reuse and test one, you have to test them all. "

Particular drivers of test costs are exactly those features which distinguish object-oriented software from procedural, problem specific software: a. encapsulation a. generalization a. association a. polymorphy

Encapsulation: a. In procedural systems a. gain access to each and every field in the storage a. all kept in a single static storage space b. fields can be set, altered and checked indiscriminately • In object-oriented systems o constructor § reason for the emphasis on built-in tests when testing classes - the test has to be part of that particular class under test § Since there is a different test for every class, that increases the test effort

Generalization: Generalization increases test effort by requiring us to test every context in which a super class can be used. a. sub classes can be added but we can't be sure that the same super class really fits to them a. the deeper the inheritance tree is, the more branches there are to test • each additional branch of the tree may have side effects on the other branches, so these too have to be retested

Association: Association allows for any object to be associated with any other object in the system. a. accomplished by invoking a foreign method in another class hierarchy a. each such association has to be tested • unlimited (spaghetti)

Polymorphy: Polymorphism makes it possible to select methods for execution at run time. a. makes the code much more flexible and general, but it is a nightmare for testing (since every possible variation of the software should be tested) a. Testing a polymorphic method invocation means testing every potential target method

Polymorphy:

Four approaches to class testing:

Non modal means testing every single method with a single object instance. Uni modal means testing every combination of methods with a single object instance. Quasi modal implies testing individual methods with multiple object instances Modal implies testing every combination of methods with multiple object instances (all possible object states) a. plus conventional challenge of testing every path through a method a. combining paths, with methods, with states leads to a combinational explosion of potential test cases

Solution: automatically generating the test cases and joining these test cases with those derived from the class specification without the support of automated class test beds there is no way to test complex classes (tools like JUnit, CPPTest and Net. Unit can fulfill this need if used correctly)

Integration Testing: a. vertical a. testing of inheritance connections • horizontal o testing associations Greatest challenge with integration testing is testing the way components interact with each other Possible kinds of component interaction: • they share a database • they send messages and receive responses • they use another components methods

Solution: The integration test of components is based on the interface descriptions. As such it is in effect an interface test. The test cases are extracted from the interface specification. a. formal interface specification language, which also includes the value domains of the arguments and result (CORBA-IDL, XML, WSDL or SQL) The goal is to test all potential combinations of arguments a. may be too many arguments to justify the costs • may be necessary to select certain representative combinations. The first approach is automatable. The second requires some form of human interaction to select combinations of states which are representative of the others.

When Agile Meets OO Testing - An Approach to testing OO Agile Projects a. Given the nature of Agile, old traditional testing using a detailed test plan isn't ideal b. To adjust to Agile and volatile requirements it becomes necessary to implement automated testing, throughout the life-cycle of a project a. automated tests allow 'fast' execution with a high return value

The Approach - Informal Reviews a. during development the developers will test their code against the requirements for the iteration/sprint b. before each release the developers and testers meet to verify the build is ready a. current sprint feature-set is checked, if there are issues they must be fixed before the build can be released

The Approach - Test Automation Pyramid Lower tier Middle tier Top tier Progressing from lower to top tests become more expensive, requiring more time and resources to complete

The Approach - Test Automation Pyramid Lower Tier a. Lowest tier consists of unit and component testing a. when needed this will include using a mock testing framework so testing can be done without relying upon back-office services b. since these can be written and maintained easily while also being run quickly they have the higher ROI a. as many tests as possible should be implemented at this tier c. All builds must pass this tier of testing before being released for further testing

The Approach - Test Automation Pyramid Middle Tier a. Middle tier consists of testing high-level business functionality and back-office services a. this is done using a tool such as JUnit, allowing testing to be done independently from the presentation layer a. this is done as it is less expensive to write and maintain b. however, these tend to be slower since they cover a large breath of functionality and accesses back-office services with delays

The Approach - Test Automation Pyramid Top Tier a. Top tier consists of testing the presentation layer using a testing framework a. this is done using a tool that typically uses low level language interfaces to trigger GUI interactioins b. given the 'brittle and expensive' nature of GUI tests there should only be a small investment in top tier testing

The Approach - Load/Stress Testing a. For systems with back-office services, testing of these services should be done directly, without integrating other development layers a. tools such a JMeter should be used a. opensource and can be extended as neccessary

The Approach - Manual Testing a. Agile projects will have manual testing as well but a. it must be minimized, allocating as few resources as possible b. it is expensive and test cases don't carry on well c. pushed as far forward a possible in the sprints a. pushed to more stable builds of the project

The Concept of "Good Enough Testing" a. Since Agile iterations are short and rework is the norm, insuring that only just enough testing is done to make the decision to continue or stop is key a. is the quality assessment sufficient b. are the costs reasonable c. can timely/wise decisions be made

Agile Testing Wrap-up a. Each iteration/sprint is tested b. Automated testing is a must c. Good Enough Testing is paramount

Conclusion: Testing OOP is problem at the unit and integration testing levels. In testing object – oriented systems, the test is moved up to a higher level of abstraction, where test automation is absolutely necessary. a. increased testing effort due to the increase in possible usages and to the greater number of potential interactions a. the increased flexibility and usability of the software under test makes it all the more important to automate - since only an automated test can cover the code in combination • by automating the test, emphasis is shifted from the manual creation of test cases to the manual creation of test scripts