Chapter 1 The Software Quality Challenge 1

Chapter 1 The Software Quality Challenge 1

The uniqueness of software quality assurance ¡ DO you think that there is a bug-free software? ¡ Can software developers warrant their software applications and their documentation from any bugs or defects ? ¡ What are the essential elemental differences between software and other industrial products such as automobiles, washing machines etc? 2

The essential differences between software and other industrial products can be categorized as follows : 1. 2. 3. Product complexity : # of operational modes the product permit. Product visibility : SW products are invisible. Product development and production process. 3

The phases at which the possibility of detecting defects in industrial products and software products: SW products do not benefit from the opportunities for detection of defects at the three phases of the production process Industrial products: ¡ ¡ ¡ Product development : QA -> product prototype Product production planning : Production - line Manufacturing : QA procedure applied Software products: ¡ ¡ ¡ Product development : QA -> product prototype Product production planning : Not required Manufacturing : Copying the product & printing copies 4

Factors affecting detecting defects in SW products VS other industrial products: Characteristic SW products Other industrial products Complexity Usually, v. complex allowing for v. large number of operational options Degree of complexity much lower Visibility Invisible, impossible to detect defects or omissions by sight ( diskette or CD storing ) Visible, allowing effective detection of defects by sight Nature of development and production process Opportunities to detect defects arise in only one phase, arise in all phases of namely product development and production 5

Important Conclusion The great complexity as well as invisibility of software, among other product characteristics, make the development of SQA methodologies and its successful implementation a highly professional challenge 6

The environment for which SQA methods are developed Pupils & students ¡ Hobbies ¡ Engineers, economics , mgt & other fields ¡ SW development professionals ¡ All those SW developers are required to deal with SW quality problems “Bugs” 7

SQA environment The main characteristics of this environment : 1. 2. 3. 4. 5. 6. 7. Contractual conditions Subjection to customer-supplier relationship Required teamwork Cooperation and coordination with other SW teams Interfaces with other SW systems The need to continue carrying out a project despite team member changes. The need to continue out SW maintenance for extended period. 8

Contractual conditions the activities of SW development & maintenance need to cope with : A defined list of functional requirements ¡ The project budget ¡ The project timetable ¡ 9

Subjection to customer-supplier relationship ¡ SW developer must cooperate continuously with customer : l l l To consider his request to changes To discuss his criticisms To get his approval for changes 10

Required teamwork ¡ Factors motivating the establishment of a project team: l l l Timetable requirements The need of variety The wish to benefit from professional mutual support & review for enhancement of project quality 11

Cooperation and coordination with other SW teams ¡ Cooperation may be required with: l l Other SW dev. Teams in the same org. HW dev. teams in the same org. SW & HW dev. teams of other suppliers Customer SW and HW dev. teams that take part in the project’s dev. 12

Interfaces with other SW Systems Input interfaces ¡ Output interfaces ¡ I/O interfaces to the machine’s control board, as in medical and lab. Control systems ¡ 13

The need to continue carrying out a project despite team member changes. ¡ During project dev. Period we might be face : l l l Leave from the members of the team Switch in employees Transfer to another city 14

The need to continue out SW maintenance for extended period. ¡ From 5 to 10 years , customers need continue to utilizing their systems: l l l Maintenance Enhancement Changes ( Modification ) 15

Chapter 2 What is Software Quality ? 16

What is Software ? ¡ IEEE Definition: Software Is : Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system. 17

IEEE Definition is almost identical to the ISO def. ( ISO/IEC 9000 -3 ) Computer programs (“Code”) ¡ Procedures ¡ Documentation ¡ Data necessary for operation the SW system. ¡ 18

TO sum up: Software quality assurance always includes : ¡ Code quality ¡ The quality of the documentation ¡ And the quality of the necessary SW data 19

SW errors, faults and failures ¡ Questions arise from HRM conference Page 16. An error : can be a grammatical error in one or more of the code lines, or a logical error in carrying out one or more of the client’s requirements. ¡ Not all SW errors become SW faults. ¡ SW failures that disrupt our use of the software. ¡ 20

The relationship between SW faults & SW failures: ¡ Do all SW faults end with SW failures? l l ¡ The answer is not necessarily The SW fault becomes a SW failure only when it is activated. Example page 17 -18 21

Classification of the causes of SW errors are the cause of poor SW quality ¡ SW errors can be ¡ l l l ¡ Code error Documentation error SW data error The cause of all these errors are human 22

The nine causes of software errors 1. 2. 3. 4. 5. 6. 7. 8. 9. Faulty requirement definition Client-developer communication failures Deliberate deviation from SW requirements Logical design errors Coding errors Non-compliance with documentation and coding instructions Shortcomings of the testing process Procedure errors Documentation errors 23

Faulty requirement definition 1. 2. 3. 4. Erroneous definition of requirements Absence of vital requirements Incomplete definition of requirements Inclusion of unnecessary requirements 24

Client-developer communication failures ¡ ¡ Misunderstandings resulting from defective client-developer comunications. Misunderstanding of the client’s requirements changes presented to the developer l l In written forms Orally Responses to the design problems others 25

Deliberate deviation from SW requirements ¡ ¡ ¡ The developer reuse SW modules taken from the earlier project Due to the time budget pressure Due to the unapproved improvements 26

Logical design errors ¡ This is come from systems architects, system analysts, SW engineers such as: l l l Erroneous algorithms Process definitions that contain sequencing errors Erroneous definition of boundary conditions Omission of required SW system states Omission of definitions concerning reactions to illegal operations 27

Coding errors ¡ ¡ Misunderstanding the design documentation Linguistic errors in the prog. Lang. Errors in the application of CASE and other dev. Tools etc 28

Non-compliance with documentation and coding ¡ ¡ ¡ Team members who need to coordinate their own codes with code modules developed by non-complying team members Individuals replacing the non-complying team member will find it difficult to fully understand his work. Design review to other non-complying team 29

Shortcomings of the testing process ¡ ¡ Incomplete testing plans Failures to document and report errors and faults Failures to promptly correct detected SW faults as a result of inappropriate indications of the reasons for the fault. Incomplete correction of detected errors. 30

Procedure errors & documentation errors ¡ See example page 22 31

Software quality - Definition IEEE 1. 2. The degree to which a system, component, or process meets specified requirements. The degree to which a system, component, or process meets customer or user needs or expectations. 32

Software Quality Pressman’s def. Conformance to explicitly stated functional and performance requirements, explicitly documented standards, and implicit characteristics that are expected of all professionally developed software. 33

Software Quality Assurance The IEEE Definition ¡ 1. 2. SQA is : A planned and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements. A set of activities designed to evaluate the process by which the products are developed or manufactured. Contrast with quality control. 34

IEEE SQA definition – exclude the maintenance & timetable and budget issues. ¡ The Author adopts the following : ¡ SQA should not be limited to the development process. It should be extended to cover the long years of service subsequent to product delivery. Adding the software maintenance functions into the overall conception of SQA. ¡ SQA actions should not be limited to technical aspects of the functional requirements, It should include activities that deal with scheduling and timetable and budget. 35

SQA – Expanded Definition A systematic, planned set of actions necessary to provide adequate confidence that the software development process or the maintenance of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating. within the budgetary confines. This definition corresponds strongly with the concepts at the foundation of ISO 9000 -3, 1997 and also corresponds to the main outlines of the CMM for software See the Table 2. 2 page 27 36

Software Quality Assurance Vs. Software Quality Control ¡ Quality Control : a set of activities designed to evaluate the quality of a developed or manufactured product. It take place before the product is shipped to the client. ¡ Quality Assurance : the main objective is to minimize the cost of guaranteeing quality by a variety of activities performed throughout the causes of errors, and detect and correct them early in the dev. Process. 37

The objectives of SQA activities see page 29 ¡ Software development ( process-oriented ) ¡ Software maintenance ( Product-oriented ) 38

SQA Vs Software Engineering ¡ 1. SW Engineering ( IEEE def. ) The application of a systematic, restricted, quantifiable approach to the development and maintenance of SW; that is the application of engineering to software. 39

Chapter 3 Software Quality Factors 40

SQ. Factors ¡ From the previous chapters we have already established that the requirements document is one of the most important elements for achieving SQ. ¡ What is a “Good” SQ requirements document ? 41

The need for comprehensive SQ requirements ¡ ¡ ¡ Our Sales IS seems v. good , but it is frequently fails, at least twice a day for 20 minutes or more. ( SW house claims no responsibility…. Local product contains a SW and every thing is ok, but, when we began planning the development of a European version, almost all the design and programming will be new. etc see page 36. 42

There are some characteristics common to all these buts : All SW projects satisfactorily fulfilled the basic requirements for correct calculations. ¡ All SW projects suffered from poor performance in important areas such as maintenance, reliability, SW reuse, or training. ¡ The cause for poor performance of the developed SW projects in these areas was lack of predefined requirements to cover these important aspects of the SW functionality. ¡ The solution is : The need for a comprehensive definition of requirements ( SQ Factors ) ¡ 43

Classification of SW requirements into SW quality factors. ¡ ¡ Mc. Call’s Factor Model This model classifies all SW requirements into 11 SW quality factors, grouped into 3 categories: l Product operation: Correctness, Reliability, Efficiency, Integrity, Usability l Product revision : Maintainability, Flexibility, Testability l Product transition : Portability, Reusability, Interoperability. See the Mc. Call model of SW quality factors tree see page 38 44

Product operation SW quality factors ¡ Correctness: Output specifications are usually multidimensional ; some common include: l l l l The output mission The required accuracy The completeness The up-to-dateness of the info. The availability of the info. ( the reaction time ) The standards for coding and documenting the SW system See Example page 39. 45

Product operation SW quality factors Reliability: Deals with failures to provide service. They determine the maximum allowed SW system failure rate, and can refer to the entire system or to one or more of its separate functions. See examples page 39 ( heart-monitoring unit ) ¡ 46

Product operation SW quality factors Efficiency: Deals with the HW resources needed to perform all the functions of the SW system in conformance to all other requirements. See examples page 40 ( CPU speed. . etc ) ¡ Integrity: Deals with the SW system security, that is requirements to prevent access to unauthorized persons. See examples page 40 ¡ 47

Product operation SW quality factors Usability: Deals with the scope of staff resources needed to train a new employee and to operate the SW system. See examples page 41 ¡ 48

Product revision SW quality factors Maintainability : Maintainability requirements determine the efforts that will be needed by users and maintenance personnel to identify the reasons for SW failures, to correct the failure, and to verify the success of the corrections. Example : Typical maintainability requirements: 1. The size of a SW module will not exceed 30 statements 2. The programming will adhere to the company coding standards and guidelines. ¡ 49

Product revision SW quality factors Flexibility : The capabilities and efforts required to support adaptive maintenance activities are covered by flexibility requirements. This factor’s requirements also support perfective maintenance activities, such as changes and additions to the SW in order to improve its service and adapt it to changes in the firm’s technical or commercial environment. Example : page 42 ¡ 50

Product revision SW quality factors ¡ - - - Testability : Deal with the testing of an IS as well as with its operation. Providing predefined intermediate results and log files. Automatic diagnostics performed by the SW system prior starting the system, to find out whether all components of SW system are in working order. Obtain a report about detected faults. Example : page 42, 43 51

Product transition SW quality factors Portability : Tend to the adaptation of a SW system to other environments consisting : ¡ - Different HW Different OS Example : SW designed to work under windows 2000 env. Is required to allow low-cost transfer to Linux. - 52

Product transition SW quality factors ¡ - - Reusability : Deals with the use of SW modules originally designed for one project in a new SW project currently begin developed. The reuse of SW is expected to save resources. , shorten the project period, and provide higher quality modules. These benefits of higher quality are based on the assumption that most SW faults have already been detected by SQA activities performed previously on it. 53

Product transition SW quality factors Interoperability : Focus on creating interfaces with other SW systems or with other equipment firmware. Example: ¡ - - The firmware of medical lab. equipment is required to process its results according to a standard data structure that can be then serve as input for a number of standard laboratory IS. 54

Alternative Models Of SW Quality Factors ¡ Two other models for SQ factors: l l ¡ Evans and Marciniak 1987 ( 12 factors ) Deutsch and Willis 1988. ( 15 factors ) Five new factors were suggested l l l Verifiability Expandability Safety Manageability Survivability 55

Alternative Models Of SW Quality Factors ¡ Five new factors were suggested l l l Verifiability: define design and programming features that enable efficient verification of the design and programming ( modularity, simplicity, adherence to documentation and prog guidelines. ) Expandability: refer to future efforts that will be needed to serve larger populations, improve services, or add new applications in order to improve usability. Safety: meant to eliminate conditions hazardous to equipment as a result of errors in process control SW. Manageability: refer to the admin. tools that support SW modification during the SW development and maintenance periods. Survivability: refer to the continuity of service. These define the minimum time allowed between failures of the system, and the maximum time permitted for recovery of service. 56

Who is interested in the definition of quality requirements ? ¡ ¡ The client is not the only party interested in defining the requirements that assure the quality of the SW product. The developer is often interested also specially : l l l ¡ Reusability Verifiability Porotability Any SW project will be carried out according to 2 requirements document : l l The client’s requirements document The developer’s additional requirements document. 57

Chapter 4 The Components Of the SQA system- Overview 58

The SQA system- an SQA architecture ¡ SQA system components can be classified into 6 classes : l l l Pre-project components Components of project life cycle activities assessment Components of infrastructure error prevention and improvement. Components of SQ management Components of standardization, certification, and SQA system assessment Organizing for SQA- the human components 59

Pre-project Components : ¡ To assure that : 1. The project commitments have been adequately defined considering the resources required, the schedule and budget. 2. The development and quality plans have been correctly determined. 60

Components of project life cycle activities assessment: ¡ The project life cycle composed of two stages: 1. The development Life cycle stage: l Detect design and programming errors l Its components divided into: l Reviews l Expert opinions l Software testing l Assurance of the quality of the subcontractors’ work and customer-supplied parts. 2. The operation-maintenance stage l Include specialize maintenance components as well as development life cycle components, which are applied mainly for functionality improving maintenance tasks. 61

Components of infrastructure error prevention and improvement : Main objectives of these components, which are applied throughout the entire organization, are : To eliminate or at least reduce the rate of errors, based on the organization’s accumlated SQA experience. 62

Components of software quality management : This class of components is geared toward several goal: The major ones being the control of development and maintenance activities and introduction of early managerial support actions that mainly prevent or minimize schedule and budget failures and their outcomes. 63

Components of standardization, certification, and SQA system assessment The main objective of this class are: 1. Utilization of international professional knowledge 2. Improvement of coordination of the organizational quality system with other organizations 3. Assessment of the achievements of quality systems according to a common scale. ¡ The various standards classified into 2 groupes: l l Quality management standards Project process standards. 64

Organizing for SQA- the human components The SQA organizational base includes : l Managers l Testing personnel l The SQA unit and practitioners interested in SQ. ¡ The main objectives are l to initiate and support the implementation of SQA components l Detect deviation from SQA procedures and methodology l Suggest improvements 65

Part II Pre-project SQ components Chapter 5 Contract Review 66

Contract Review Is the software quality element that reduces the probability of undesirable situation like in ( CFV project ). Contract review is a requirement by the ISO 9001 and ISO 9000 -3 guidelines. 67

The Contract review process and its stages ¡ Several situations can lead a SW company to sign a contract with a customer such as : l l Participation in a tender Submission of a proposal according to the customer’s RFP. Receipt of an order from a company’s customer Receipt of an internal request or order from another department in the organization 68

The Contract review process and its stages ¡ Contract review : l is the SQA component devised to guide review drafts of proposal and contract documents. l If applicable, provides oversight ( supervision ) of the contracts carried out with potential project partners and subcontractors. 69

The Contract review process itself is conducted in two stages : ¡ Stage 1 – Review of the proposal draft prior to submission to the potential customer ( proposal draft review ): Reviews the final proposal draft and proposal’s foundations: l l Customer’s requirement documents Customer’s additional details and explanations of the requirements Cost and resources estimates Existing contracts or contract drafts of the supplier with partners and subcontractors. 70

The Contract review process itself is conducted in two stages : ¡ Stage 2 – Review of the proposal draft prior to signing ( Contract draft review ): Reviews the contract draft on the basis of the proposal and the understandings ( include changes ) reached during the contract negotiations sessions. ¡ The individuals who perform the review thoroughly examine the draft while referring to a comprehensive range of review subjects ( a Check-list ) is very helpful for assuring the full coverage of relevant subjects. See appendix 5 A, 5 B ¡ 71

Contract Review objectives: ¡ Proposal draft review objectives( assure the following ) l l l l Customer requirements have been clarified and documented Alternative approaches for carrying out the project have been examined Formal aspects of the relationship between the customer and SW firm have been specified. Identification of development risks Adequate estimation of project resources and timetable have been prepared. Examination of the customer’s capacity to fulfill his commitments Definition of partners and subcontractors participation conditions Definition and projection proprietary rights. 72

Contract Review objectives: ¡ Contract draft review objectives( assure the following ) l l l No un-clarified issues remain in the contract draft All the understandings reached between the customer and the firm are to be fully and correctly documented. No changes, additions, or omissions that have not been discussed and agreed upon should be introduced into contract draft. 73

Factors affecting the extent of a contract review: ¡ ¡ Project magnitude, usually measured in man-month resources. Project technical complexity Degree of staff acquaintance with and experience in the project area. Project organizational complexity, the greater the number of organizations ( partners, subcontractors, and customers ) taking part in the project, the greater the contract review efforts required. 74

Who performs a contract review: ¡ ¡ The leader or another member of the proposal team The members of the proposal team An outside professional or a company staff member who is not a member of the proposal team. A team of outside experts. 75

Implementation of a contract review of a major proposal ¡ The characteristics of the major proposal : l l ¡ Very large-scale project Very high technical complexity New professional area for the company High organizational complexity The difficulties of carrying out contract reviews for major proposals : l l l Time pressures Proper contract review requires substantial professional work The potential contract review team members are very busy. 76

Implementation of a contract review of a major proposal ¡ Recommended avenues ( approaches ) for implementing major contract reviews : l l The contract review should be scheduled. A team should carry out the contract review A contract team leader should be appointed The activities of the team leader include : ¡ ¡ ¡ Recruitment of the team members Distribution of review tasks Coordination between members Coordination between the review team and the proposal team Follow-up of activities, especially compliance with the schedule Summarization of the findings and their delivery to the proposal team. l 77

Contract review for internal projects ¡ ¡ See table 5. 1 page 86 The main point here is the internal relationship. Loose relationships are usually characterized by insufficient examination of the project’s requirements, its resources and development risks. To avoid the previous problems we have to apply the contract review to the internal as external projects by implementing procedures that define : l l l An adequate proposal for the internal project Applying a proper contract review process An adequate agreement between the internal customer and the internal supplier. 78

Chapter 6 Development and quality plans ¡ ¡ ¡ Development plans and quality plans are the major elements needed for project compliance with ISO 9000. 3 standards and ISO/IEC 2001 and with IEEE 730. It is also an important element in the Capability Maturity Model ( CMM ) for assessment of SW development organization maturity. The projects needs development and quality plans that : l l Are based on proposal materials that have been re-examined and thoroughly updated Are more comprehensive than the approved proposal, especially with respect to schedules, resources, estimates, and development risk evaluations Include additional subjects, absent from the approved proposal others 79

Development plan and quality plan objectives 1. 2. 3. 4. 5. Scheduling development activities that will lead to successful and timely completion of the project, and estimating the required manpower resources and budget. Recruiting team members and allocating development resources. Resolving development risks. Implementing required SQA activities Providing mgt. with data needed for project control. 80

Elements of the development plan 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Project products Project interfaces Project methodology and development tools SW development standards and procedures The mapping of the development process. ( proj. mgt. Gant ) Project milestones ( documents , code , report ) Project staff organization ( org. stru. , prof. req. , no of team mem. , names of team leaders ) Development facilities ( SW, HW tools, space, period req. for each use ) Development risks ( see next slide ) Control methods Project cost estimation 81

Development risks ¡ 1. 2. 3. Is a state or property of a development task or environment which, if ignored, will increase the likelihood of project failure. Such as : Technological gap Staff shortages Interdependence of organizational elements- the likelihood that suppliers or specialized HW or SW subcontractors, for example, will not fulfill their obligations or schedule. 82

Elements of Quality Plan 1. 2. 3. Quality goals ( quantitative measures example page 102 ) Planned review activities ¡ The scope of review activity ¡ The type ¡ The schedule ( priorities ) ¡ The specific procedure to be applied ¡ Who responsible for carrying out the rev. act. Planned SW tests ( a complete list of planned SW tests should be provided ) each test ¡ The unit, integration or the complete system to be tested ¡ The type of testing activities to be carried out ¡ The planned test schedule ¡ The specific procedure ¡ Who responsible 83

Elements of Quality Plan 4. Planned acceptance tests for externally developed SW 5. Configuration management configuration mgt tools and procedures, including those change-control procedures meant to be applied throughout the project 84

Dev. And Quality Plan for small projects & internal projects ¡ See page 105 , 106 85

Chapter 7 Integrating Quality activities in the project life cycle ¡ Classic and Other SW development Methodologies: l SDLC ( Req. def. , Analysis, Design, Coding, sys. Tests, install and conversion, op. and maintenance ) 86

Integrating Quality activities in the project life cycle Prototyping 87

Integrating Quality activities in the project life cycle The Spiral model See page 128 ¡ It is an improved metho. for overseeing large and more complex projects ¡ Combines SDLC & prototyping ¡ At each iteration of the spiral the following activities are performed: l l Planning Risk analysis and reslution Engineering activities Customer evaluation, comm, changes, etc 88

Integrating Quality activities in the project life cycle The object-oriented model. ¡ Easy integration of existing sw modules ( Objects ) into newly developed sw sys. ¡ A SW component library serves this purpose by supplying sw components for reuse. See page 130 ¡ Advantages of library reuse: l l l ¡ Economy Improve quality Shorter development time The advantages of OOPS will grow as the storage of reusable SW grows ( Example Microsoft and Unix ) 89

Factors affecting intensity of quality assurance activities in the development projects ¡ ¡ Quality assurance activities will be integrated into development plan that implements one or more SW development models Quality assurance planners for project are required to determine : l l The list of QA activities needed for a project For each QA activity: ¡ ¡ ¡ l Timing Who perform & the resources required Team members, external body for QA Resources required for removal of defects and introduction of changes. 90

Factors affecting intensity of quality assurance activities in the development projects ¡ Project factors l l ¡ Team factors l l ¡ Magnitude of the project Technical complexity and difficulty Extent of reusable SW components Severity of failure outcome if the project fails Professional qualification of team members Team acquaintance with the project and its experience in the area Availability of staff members who can professionally support team Familiarity with team members, in other words the percentage of new staff members in the team See example page 132 91

Verification, Validation and Qualification ¡ ¡ Three aspects of quality assurance of the SW product are examined under the issues of verification, validation, and qualification( IEEE std 610. 12 -1990) Verification : the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. l l It examines the consistency of the products being developed with products developed in the previous phases. Examiner can assure that development phases have been completed correctly 92

Verification, Validation and Qualification ¡ Validation : the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. l l It represents the customer’s interest by examining the extent of compliance to his original req. Comprehensive validation reviews tend to improve customer satisfaction from the system 93

Verification, Validation and Qualification ¡ Qualification : the process used to determine whether a system or component is suitable for operational use. l It focuses on operational aspects, where maintenance is the main issues l Planners are required to determine which of these aspects should be examined in each quality assurance activity. 94

A model for SQA defect removal effectiveness & Cost ¡ The model deals with 2 quantitative aspects : 1. 2. ¡ Effectiveness in removing project defects The cost of removal See page 135 95

Defect removal effectiveness ¡ ¡ ¡ It is assumed that any SQA activity filters ( screens ) a certain percentage of existing defects. In most cases the percentage of removed defects is somewhat lower than the percentage of detected defects as some corrections are ineffective or inadequate. The next SQA activity will faces both the remaining defects and the new defects created in the current development phases. It is assumed that the filtering effectiveness of accumulated defects of each QA activity is not less than 40%. Table 7. 4 page 136 list the average filtering effectiveness by QA activities. 96

Cost of defect removal ¡ ¡ ¡ The cost of defect removals varies by development phase, while costs rise substantially as the development process proceeds. Example : removal of a design defect detected in the design phase may require an investment of 2. 5 working days; removal of the same defect may required 40 days during the acceptance tests. Defect-removal costs based on some surveys are shown in table 7. 5 page 137. 97

The Model ¡ The model is based on the following assumptions: l l l The development process is linear and sequential, following the waterfall model. A number of new defects are introduced in each development phase ( see table 7. 3 page 135 ). Review and test SQA activities serve as filters, removing a percentage of the entering defects and letting the rest pass to the next phase. If we have 30 defects and the filtering efficiency 60% then 18 defects will be removed & 12 will stay to the next. At each phase the incoming defects are the sum of defects not removed together with the new defects introduced ( created ) in the current development phase. The cost is calculated for each QA activity by multiplying the number of defects removed by the relative cost of removing a defect. ( table 7. 5 ) The remaining defects passed to the customer, will be detected by him. 98

The model presents the following POD : phase originated defects ( table 7. 3 ) ¡ PD : passed defects. ¡ %FE : % filtering effectiveness ( table 7. 4 ) ¡ RD : removed defects ¡ CDR : cost of defect removal ( table 7. 5 ) ¡ TRC : total removal cost. ¡ 99

Chapter 8 Reviews ¡ IEEE definition Review process : A process or meeting during which a work product or set of products is presented to project personnel , managers, users, customers, or other interested parties for comment or approval. 100

Methodologies for reviewing documents ¡ ¡ Reviews acquire special importance in the SQA process because they provide early direction and prevent the passing of design and analysis errors “down-stream” , to stages where error detection and correction are much complicated and costly : The methodologies for reviewing : l l l ¡ Formal design review Peer reviews ( inspections and walkthroughs ) Expert opinions Standards for SW reviews are the subject of IEEE std 1028 ( IEEE, 1997 ). 101

Reviews Objectives ¡ ¡ ( Direct Objectives ) To detect analysis & design errors as well as subjects where corrections, changes and completions are required with respect to the original specifications and approved changes. To identify new risks likely to affect completion of the project. To locate deviations from templates and style procedures and conventions. Correction of these deviations is expected to contribute to improved communication & coordination resulting from greater uniformity of methods & documentation style. To approve the analysis or design product. Approval allows the team to continue to the next development phase. 102

Reviews Objectives ( Indirect Objectives ) ¡ To provide an informal meeting place for exchange of professional knowledge about development methods, tools, and techniques. ¡ To record analysis and design errors that will serve as a basis for future corrective actions. The corrective actions are expected to improve development methods by increasing effective and quality, among other product features. 103

Formal design reviews ( DRs ) ¡ Formal design review, also called l l ¡ ¡ Design reviews ( DRs ) Formal technical reviews ( FTR ) Without this approval, the development team cannot continue to the next phase of SW development project. Formal design review can be conducted at any development milestone requiring completion of an analysis or design document, whether that document is a requirement specification or an installation plan. 104

A list of common Formal design reviews : ¡ ¡ ¡ ¡ DPR - development plan review SRSR- Software requirement specification review PDR – Preliminary design review DDR – Detail design review DBDR – Data base design review TPR – Test plan review STPR – Software test procedure review VDR – Version description review OMR – operator manual review SMR – Support manual review TRR – Test readiness review PRR – Product release review IPR – Installation Plan review 105

The Formal Design Review will focus on : The participants ¡ The prior preparations ¡ The DR session ¡ The recommended post-DR activities ¡ 106

The participants in a DR ¡ ¡ ¡ All DRs are conducted by l A review leader l A review team The review leader: characteristics l Knowledge & experience in development of projects of the type reviewed. l Seniority at a level similar to if not higher than that of the project leader l A good relationship with the project leader and his team l A position external to the project team. Small dev. Departments and software houses typically have difficulties finding an appropriate candidate to lead the review team. One possible solution to this is the appointement of an external consultant. 107

The Review Team It is desirable for non-project staff to make up the majority of the review team. ¡ The size of the review team from 3 to 5 to be an efficient team ¡ 108

Preparation for a DR ¡ A DR session are to be completed by all three main participants in the review : l l ¡ ¡ Review leader , an team Development team. Each one is required to focus on distinct aspects of the process. Review leader preparations (main tasks) : l l l To appoint the team members To schedule the review sessions To distribute the design document among team members ( hard copy, electronic copy etc ) 109

Preparation for a DR ¡ Review team preparations (main tasks) : l l l ¡ Review the design document and list their comments prior to the review session team members may use a review checklists. See chapter 15 ( checklists ) Development team preparations ( main tasks ) l l Prepare a short presentation of the design document The presentation should focus on the main professional issues awaiting approval rather than wasting time on description of the project in general. 110

The DR session ¡ The agenda is the issue ( a typical DR session agenda ) : 1. 2. 3. 4. A short presentation of the design document Comments made by members of the review team. Verification and validation in which each of the comments is discussed to determine the required actions ( corrections, changes and addition ) that the project team has to perform. Decisions about the design product ( document ), which determines the project’s progress. These decisions take the following three forms: 111

Decisions forms : ¡ ¡ ¡ Full approval : enables immediate continuation to the next phase. It may be accompanied by demands for some minor corrections to be performed by project team. Partial approval: approval of immediate continuation to the next phase for some parts of the project, with major action items demanded for the remainder of the project. Denial of approval : demands to repeat of the DR 112

The DR report see appendix 8 A ¡ ¡ ¡ one of the review leader responsibilities is to issue a DR report immediately after the review session. The development team should perform the corrections earlier and minimize the attendant delays to the project schedule. The report major sections contain : l l A summary of the review discussion The decision of the continuation of the project A full list of the required actions ( corr, changes, additions) and the anticipated completion dates. The name(s) of the review team member(s) assigned to follow up performance of corrections. 113

The follow-up process The review leader himself is required to determine whether each action item has been satisfactory accomplished as a condition for allowing the project to continue to the next phase. ¡ Follow-up should be documented to enable clarification. ¡ 114

Pressman (2000, chapter 8 ) ¡ ¡ Pressman’s 13 “golden guidelines “ for a successful design review: See page 157 115

Peer Reviews two review methods ( Inspection and Walkthrough ) ¡ ¡ ¡ The major difference between formal design reviews and peer review methods is rooted in participants & authority. In peer reviews, as expected, the project leader’s equals, members of his department and other units. The other difference lies in degree of authority & the objective of each review method. The peer review main objectives lies in detecting errors & deviations from standards. The appearance of the CASE tools reduce the value of manual reviews such as inspection and walkthrough. Researches find out that peer reviews are highly efficient as well as effective method. 116

Inspection & Walkthrough What differentiates a walkthrough from an inspection is the level of formality, inspection is the more formal of two. ¡ Inspection emphasizes the objective of corrective actions. ¡ Walkthrough’s findings are limited to comments on the document reviewed. ¡ 117

Inspection & Walkthrough ¡ Inspection is usually based on a comprehensive infrastructure, including : l l Development of inspection checklists developed for each type of design document as well as coding language and tool, which are periodically updated. Development of typical defect type frequency tables, based past findings, to direct inspectors to potential “defect concentration areas”. Periodic analysis of the effectiveness of past inspections to improve the inspection methodology Introduction of scheduled inspections into the project activity plan and allocation of the required resources, including resources for correction of detected defects. 118

Participants of peer reviews ¡ A review leader l ¡ The author l ¡ Main tasks & qualification page 161 Invariably a participant in each type of peer review. Specialized professional l For inspections: ¡ ¡ ¡ l A designer A coder or implementer A tester For walkthrough: ¡ ¡ ¡ A standards enforcer A maintenance expert A user representative. 119

Team assignments The presenter ¡ The Scribe ¡ 120

Preparations for a peer review session Leader preparation ¡ Team’s preparation ¡ 121

Session Documentation ¡ Inspection session findings report l ¡ Inspection session summary report l ¡ Prepared by the scribe Prepared by the leader See appendix 8 b , 8 c 122

Post-Peer review activities ¡ Post-inspection activities are conducted to attest to : l l ¡ The prompt, effective correction and reworking of all errors by the designer/author and his team, as performed by the inspection leader in the course of the assigned follow-up activities. Transmission of the inspection reports to the internal Corrective Action Board ( CAB ) for analysis. See Fig 8. 2 ( comparison of the peer review methods ( Page 166 ) 123

The efficiency of peer reviews ¡ Some of the more common metrics applied to estimate the efficiency of peer reviews: l l l peer review detection efficiency( average hrs worked per defect detected) Peer review defect detection density ( average number of defects detected per page of the design document ) Internal peer review effectiveness ( % of defects detected by peer reviews as % of total defects detected by the developer). 124

Comparisons ¡ See tables page 167 -169 125

Expert opinions ( external ) ¡ It is good in the following situations : l l Insufficient in-house proff. Temporary lack in-house proff. Disagreements In small organizations 126

Chapter 9 Software testing - Strategies ¡ Testing Definition : l ¡ Testing is the process of executing a program with intention of finding errors. IEEEdefinition: 1. 2. The process of operating a system or component under specified condition, observing or recording the results, and making an evaluation of some aspect of the system or component. The process of analyzing a software item to detect the difference between the existing and required conditions ( that is, bugs ) and evaluate the features of the software item. 127

Software testing - Definition Is a formal ( SW test plan ) process carried out by specialized testing team ( independent ) in which a software unit, several integrated software units or entire software package are examined by running the programs on a computer. All the associated tests are performed according to approved test procedures on approved test case. 128

Software testing objectives ¡ Direct objectives: l l l ¡ To identify and reveal as many errors as possible in the tested SW. To bring the tested SW, after correction of the identified errors and retesting, to an acceptable level of quality. To perform the required tests efficiently, within budgetary and scheduling limitations. Indirect objectives l To compile a record of SW errors for use in error prevention ( by corrective & preventive actions ) 129

Software testing Strategies To test the SW in its entirety, once the completed package is available; otherwise known as “big bang testing “. ¡ To test the SW piecemeal, in modules, as they are completed ( unit tests ) ; then to test groups of tested modules integrated with newly completed modules ( integrated tests ). This process continues until all the entire package is tested as whole ( system test ). This testing strategy is usually termed “incremental testing” ¡ 130

Incremental testing is also performed according to two basic Strategies: Bottom-Up ( 4 stages ) see fig page 183 ¡ Top-down ( 6 stages ) ¡ The incremental pathes: ¡ l l Horizontal sequence ( breadth first ) Vertical sequence ( Depth first ) 131

Stubs & Drivers for incremental testing Stubs and drivers are SW replacement simulators required for modules not available when performing a unit test or an integration test. ¡ Stubs ( often termed a “Dummy Module “ )replaces an unavailable lower level module, subordinate to the module tested. ¡ It is required for top-down testing of incomplete systems. See example fig 9. 2 ¡ 132

Stubs & Drivers for incremental testing A driver is a substitute module but of the upper level module that activates the module tested. ¡ The driver is passing the test data on to the tested module and accepting the results calculated by it. ¡ It is required in bottom-up testing until the upper level modules are developed. ¡ See example fig 9. 2 ¡ 133

Bottom-Up Vs Top-Down strategies ¡ Main Adv. Of bottom-up : l ¡ Main disadv. l ¡ The lateness at which the prog. As whole can be observed ( at the stage following testing of the last module ) Main adv. Of Top-down: l ¡ The relative ease of its performance The possibility it offers to demonstrate the entire prog. Function shortly after activation of the upper-level modules has been completed. This characteristic allows for early identification of analysis & design errors related to algorithms, functional requir. , and the like. Main disadv. l l The relative difficulty of preparing the required stubs, with often require very complicated programming. The relative difficulty of analyzing the result of the tests. 134

Big bang Vs. Incremental testing ¡ The main disadvantages of big bang : l l l ¡ Identification of errors becomes difficult Corrections will be at the same time. Error correction estimation of the required testing resources and testing schedule a rather fuzzy endeaver. Incremental testing adv. : l l Usually performed on relatively small SW modules, as unit or integration tests. ( more errors detection ) Identification and correction of errors is much simpler and requires fewer resources because it is performed on a limited volume of SW. 135

Software test classification. Classification according to testing concept: ¡ Two testing classes have been developed: l Black box ( functionality ) testing: Identifies bugs only according to SW malfunctioning as they are revealed in its erroneous output. ¡ Incases that outputs are found to be correct, black box testing disregarded the internal path of calculations and processing performed. ¡ l White box ( structural ) testing: Examines internal calculation paths in order to identify bugs. ¡ The white is meant to emphasize the contrast between methods ¡ 136

Software test classification. Classification according to requirements: ¡ See table 9. 1 Page 188 137

White Box Testing ¡ ¡ White box testing concept requires verification of every program statement and comment. See table 9. 2 , white box testing enables performance of l l ¡ ¡ data processing and calculations correctness tests SW qualification tests Maintainability tests Reusability tests Every computational operation in the sequence of operations created by each test case ( “Path” ) must be examined. This type of verification allows us to decide whether the processing operations and their sequences were programmed correctly for the path in question, but not for other pathes. 138

Data processing and calculation correctness tests Path coverage & line coverage ¡ Path coverage : total number of possible paths l ¡ ¡ 10 if-then-else : 1024 paths Line coverage: for full line coverage, every line of code be executed at least once during the process of testing. The line coverage metrics for completeness of a linetesting plan are defined as the percentage of lines indeed executed during the tests. Flow chart and a program flow graph are used. Example 191. 139

Mc. Cabe’s cyclomatic complexity metrics ¡ ¡ ¡ Measures the complexity of a program or module at the same time as it determines the maximum number of independent paths needed to achieve full line Coverage of the program. The measure is based on graph theory using program flow graph. An independent path is defined with reference to the succession of independent paths accumulated. Independent path is any path on the program flow graph that includes at least one edge that is not included in any former independent paths. See table 9. 5 page 194 140

Mc. Cabe’s cyclomatic complexity metrics ¡ ¡ ¡ The cyclomatic complexity metric V(G) = R = E – N + 2 = P + 1 Where R = the number of regions E = number of edges N = number of nodes P = number of decisions ( nodes having more than one leaving edge). See example page 195 An empirical studies show that if V(G) < 5 considered simple If it is 10 or less considered not too difficult If 20 or more it is high If it is exceed 50 the SW for practical purposes becomes untestable. 141

SW qualification & reusability testing SW qualification testing ¡ SW reusability testing ¡ Adv & disadv. Of White box ¡ Page 197 ¡ 142

143