Software Testing and Quality Assurance Theory and Practice

Software Testing and Quality Assurance Theory and Practice Chapter 7 System Integration Testing Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 1

Outline of the Chapter • • • The Concept of Integration Testing Different Types of Interfaces Different Types of Interface Errors Granularity of System Integration Testing System Integration Techniques: Incremental, Top-down, Bottom-up, and Sandwich and Big-bang Software and Hardware Integration Hardware Design Verification Tests Hardware and Software Compatibility Matrix Test Plan for System Integration Off-the-self Component Integration Off-the-shelf Component Testing Built-in Testing Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 2

The Concept of Integration Testing • • • A software module contained element of a system which can be a subroutine, function, procedure, class, or collection of those basic elements put together to deliver a higher level service. Modules are individually tested, which is commonly known as unit testing. put the modules, that is, pieces, together to construct the complete system. Constructing a working system from the pieces is not a straightforward task, because of numerous interface errors. Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 3

The Concept of Integration Testing • • • System Integration Testing : Modules are assembled to construct larger subsystem and tested A software module is a self-contained element of a system Modules are individually tested commonly known as unit testing (subroutine, function, procedure or class) Next major task is to put the modules, i. e. , pieces together to construct the complete system Construction of a working system from the pieces is not a straightforward task because of numerous interface errors Integration testing is said to be complete when – The system is fully integrated together – All the test cases have been executed – All the defects found have been fixed Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 4

The Concept of System Integration Testing The major advantages of conducting SIT are as follows: 1. Defects are detected early 2. It is easier to fix defects detected earlier 3. We get earlier feedback on the acceptability of the individual modules and on the overall system 4. Scheduling of defect fixes is flexible, and it can overlap with development Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 5

Different Types of Interfaces Interface between two modules allows one module to access the service provided by another module. Three common paradigms for interfacing modules: 1. Procedure call interface: It implements a mechanism for passing control and data between modules. 2. Shared memory interface: a block of memory is shared between two modules. The memory block may be allocated by one of the two modules. Data are written into the memory block by one module and are read from the block by the other. 3. Message passing interface: one module prepares a message and sending it to another module. This form of module interaction is common in client-server system. The problem arises when we “put modules together” because of interface errors Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 6

Different Types of Interface Errors 1. Construction 9. Inadequate error processing 2. Inadequate functionality 10. Additions to error processing 3. Location of functionality 11. Inadequate post-processing 4. Changes in functionality 12. Inadequate interface support 5. Added functionality 13. Initialization/value errors 6. Misuse of interface 14. Validation od data constraints 7. Misunderstanding of interface 15. Timing/performance problems 8. Data structure alteration 16. Coordination changes 17. Hardware/software interfaces Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 7

Different Types of Interface Errors 1 - Construction: Some programming languages, such as C, generally separate the interface specification from the implementation code. In a c program , programmers can write a statement #include header. h, where header. h contains an interface specification. Since the interface specification lies somewhere away from the actual, programmers overlook the interface specifications while writing code. Therefore, inappropriate use of #include statements cause construction error. 4 - Changes in functionality: Changing one module without correctly adjusting for that change in the other related modules affects the functionality of the program. 9 - Inadequate error processing : A called module may return an error code to the calling module. However, the calling module may fail to handle the error properly. Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 8

Granularity of System Integration Testing is performed at different levels of granularity Granularity: it extend to which a system is composed of distinguishable pieces. 1. Intra-system testing This form of testing constitutes low-level integration testing with the objective of combining the modules together to build a cohesive system 2. Inter-system testing – It is a high-level testing phase which requires interfacing independently tested systems – all the systems are connected together, and testing is conducted from end to end. 3. Pairwise testing – In pairwise integration, only two interconnected systems in an overall system are tested at a time – The purpose of pairwise testing is to ensure that two systems under consideration can function together. Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 9

System Integration Techniques Common approaches to perform system integration testing 1. Incremental 2. Top-down 3. Bottom-up 4. Sandwich 5. Big-bang Pre-requisite A module must be available to be integrated A module is said to available for combining with other modules when the module’s check-in request form is ready Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 10

Check-in Request Form Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 11

Incremental • • Integration testing is conducted in an incremental manner as a series of test cycles In each test cycle, a few more modules are integrated with an existing core modules and developers fix all the errors found, and continue the next cycle of testing The complete system is built incrementally, cycle by cycle until the whole system is operational for system-level testing. The number of SIT cycles and the total integration time are determined by the following parameters: – – – Number of modules in the system Relative complexity of the module (cyclomatic complexity) Relative complexity of the interfaces between the modules Number of modules needed to be clustered together in each test cycle Whether the modules to be integrated have been adequately tested before Turnaround time for each test-debug-fix cycle Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 12

Incremental Creating a daily build is very popular among many organization because: • • • It facilitates to a faster delivery of the system It puts emphasis on small incremental testing It steadily increases number of test cases The system is tested using automated, re-usable test cases An effort is made to fix the defects that were found within 24 hours Prior version of the build are retained for references and rollback Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 15

Top-down • • • 1. Figure 7. 1: A module hierarchy with three levels and seven modules 2. Module A has been decomposed into modules B, C, and D (7. 1) Modules B, D, E, F, and G are terminal modules (7. 1) Process of Top-down approch: First integrate modules A and B using stubs C` and D` (represented by grey boxes) (7. 2) Next stub D` has been replaced with its actual instance D (7. 3) – Two kinds of tests are performed: • Test the interface between A and D • Regression tests to look for interface defects between A and B in the presence of module D Figure 7. 2: Top-down integration of modules A and B Figure 7. 3: Top-down integration of modules A, B and D Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 16

Top-down 4. Stub C` has been replaced with the actual module C, and new stubs E`, F`, and G` (7. 4) – Perform tests as follows: • first, test the interface between A and C; • second, test the combined modules A, B, and D in the presence of C 5. The rest of the process depicted in the right hand side figures. (7. 5, 7. 6) Figure 7. 4: Top-down integration of modules A, B, D and C Figure 7. 5: Top-down integration of modules A, B, C, D and E Figure 7. 6: Top-down integration of modules A, B, C, D, E and F Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 17

Top-down Advantages • The SIT engineers continually observe system-level functions as the integration process continue • Isolation of interface errors becomes easier because of the incremental nature of the top-down integration • Test cases designed to test the integration of a module M are reused during the regression tests performed after integrating other modules Disadvantages • It may not be possible to observe meaningful system functions because of an absence of lower level modules and the presence of stubs. • Test case selection and stub design become increasingly difficult when stubs lie far away from the top-level module. Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 18

Bottom-up 1. 2. 3. 4. 5. 6. We design a test driver to mimic module C and integrate lowest-level modules E, F, and G (7. 8) Return values generated by one module is likely to be used in another module(7. 8) The test driver is replaced with actual module , i. e. , C. (7. 9) A new test driver is used to mimic the behavior of module A (7. 9) At this moment, more modules such as B and D are integrated (7. 9) Finally, the test driver is replaced with module A and further test are performed (7. 10) Figure 7. 8: Bottom-up integration of module E, F, and G Figure 7. 9: Bottom-up integration of module B, C, and D with F, F, and G Figure 7. 10 Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 19

Bottom-up Advantages • If the low-level modules and their combined functions are often invoked by other modules, then it is more useful to test them first so that meaningful effective integration of other modules can be done Disadvantages • Discovery of major faults in system design may not be possible until the top-level modules. • Test engineers can not observe system-level functions from a partly integrated system. In fact, they can not observe system-level functions until the top-level test driver is in place Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 20

Big-bang and Sandwich Big-bang Approach • First all the modules are individually tested • Next all those modules are put together to construct the entire system which is tested as a whole Sandwich Approach • In this approach a system is integrated using a mix of top-down, bottom-up, and big-bang approaches • A hierarchical system is viewed as consisting of three layers • The bottom-up approach is applied to integrate the modules in the bottom-layer • The top layer modules are integrated by using top-down approach • The middle layer is integrated by using the big-bang approach after the top and the bottom layers have been integrated Software Testing and QA Theory and Practice (Chapter 7: System Integration Testing) © Naik & Tripathy 21