CS 425 625 Software Engineering Verification and Validation Based

CS 425/625 Software Engineering Verification and Validation Based on Chapter 19 of the textbook [SE-6] Ian Sommerville, Software Engineering, 6 th Ed. , Addison-Wesley, 2000 and on the Ch 19 Power. Point presentation available at the book’s web-site: www. comp. lancs. ac. uk/computing/resources/Ian. S/SE 6/Slides/index. html November 10, 2003 1

Outline n n n 2 Introduction Testing and Debugging Verification and Validation Planning Software Inspections Program Inspections (forms of Software Inspections) Automatic Static Analysis

Introduction…. . n n n 3 Verification and validation (V&V): the checking and analysis processes that ensure the software satisfies its specification and meets the needs of the clients who are paying for it Validation: “Are we building the right product? ” Verification: “Are we building the product right? ” Verification involves checking the software conforms with its specification while the more general process of validation ensures the software meets the needs of the clients V&V is a whole life-cycle process, encompassing requirements reviews, design reviews, code inspections, and program testing

. Introduction. … n V & V techniques: u Software inspections u Software testing n Software inspections u u u 4 Are static V&V techniques as they do not require the software to be executed Consist of inspections, automated static analyses, and formal verifications of source code or system models Can only check the correspondence between the software and its specification Cannot demonstrate the system is operationally useful Cannot check non-functional requirements of the software

. . Introduction. . . n 5 Software testing u It is a dynamic V&V technique as it needs an executable version of the software system u The system is executed with test data and its operational behaviour is assessed u Can reveal the presence of errors, not their absence u A successful test is a test which discovers one or more errors u The V&V technique for checking non-functional requirements and for verifying system integration u Should be used in conjunction with static techniques to provide full V&V coverage

…Introduction. . n Types of software testing u Defect Ø Ø Ø testing Intended to find inconsistencies between a program and its specification Tests designed to discover program faults and defects A successful defect test is one which reveals the presence of defects in a system u Statistical Ø Ø 6 testing Designed for software’s performance and reliability estimation By running tests that reflect actual user inputs and their frequency, an estimate of operational reliability can be made

…. Introduction. n 7 Static and dynamic V&V [Fig 19. 1, Somm 00]

…. . Introduction n Verification and validation should establish confidence that the software is fit for purpose This does not mean the software is completely free of defects; rather, it must be good enough for its intended use The required level of confidence depends on: Software’s purpose: the level of confidence depends on how critical the software is to an organisation u User expectations: users may have lower expectations of certain types of software u Marketing environment: getting a product to market early may have higher priority than finding its defects u 8

Testing and debugging. n n 9 Defect testing and debugging are distinct processes V&V is concerned with establishing the existence of defects in a program Debugging is concerned with locating and repairing these errors Debugging involves formulating hypotheses about program behaviour then testing these hypotheses to find the errors

. Testing and debugging The debugging process [Fig. 19. 2, Somm 00] 10

V & V Planning. . n n 11 The planning of V&V should start early in the development process The plan should balance static verification and testing, specify testing standards and procedures, establish checklists for inspections, and define the software test plan Test planning breaks down V&V into a number of stages, often organized according to the V-model (shown on next page) The focus is on setting standards and procedures for inspections and testing, not on describing product tests

. V&V Planning. The V-model of development [Fig. 19. 3, Somm 00] 12

. . V&V Planning n The structure of a software test plan u u u u 13 The testing process Requirements traceability Tested items Testing schedule Test recording procedures Hardware and software requirements Constraints

Software Inspections. n n 14 Involve people examining software code or models (representations) with the aim of discovering defects Do not require execution of a system thus can be used throughout the development process May be applied to any representation of the system: requirements, design, test data, etc. Very effective technique for discovering errors

. Software Inspections n n 15 In software inspections many different defects can be discovered in a single review of the source code or software model In testing, one defect may mask another hence several executions are required Software inspections reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly occur Software inspections and software testing are complementary, not competing techniques (see also slides 4 and 5)

Program Inspections……. n n n 16 Program inspections are a type of software inspections Consist of formal reviews conducted by teams and intended for program defect detection Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e. g. , an un-initialized variable), or non-compliances with standards

. Program Inspections…… n Inspection pre-conditions: u u u 17 A precise specification must be available Team members must be familiar with the organisation standards Syntactically correct code must be available An error checklist should be prepared Management must accept that inspection will increase costs early in the software process Management must not use inspections for staff appraisal

. . Program Inspections…. . The inspection process [Fig. 19. 6, Somm 00] 18

…Program Inspections…. n Program inspection procedure: u u u 19 The inspection is planned by the moderator (or chairman) The system overview is presented to the inspection team by the program author (or owner) The code and associated documents are distributed to the inspectors (inspection team) in advance for individual preparation The inspection meeting takes place and errors are noted (the inspection also involves a reader and, possibly, a scribe) During re-work modifications are made by the author to repair discovered errors Re-inspection may or may not be required, based on moderator’s decision

…. Program Inspections… n Inspection teams are made up of at least 4 members: u u u 20 Author of the code being inspected Inspector who finds errors, omissions and inconsistencies Reader who reads the code to the team Moderator who chairs the meeting and notes discovered errors Other roles are chief moderator and scribe

…. . Program Inspections. . n Inspection checklist u u 21 A checklist of common errors should be used during the inspection This error checklist is programming language dependent The weaker the type-checking of the language, the larger the checklist Examples of possible errors: initialisation, constant naming, loop termination, array bounds, etc.

Inspection checks (fault classes) [Fig. 19. 7, Somm 00] 22

……. Program Inspections n Inspection rates: u u u 23 500 statements/hour during overview 125 source statement/hour during individual preparation 90 -125 statements/hour can be inspected Inspection is therefore an expensive process Inspecting 500 lines costs about 40 man/hours effort = £ 2800

Automated Static Analysis…. n n n 24 Static analyzers are software tools for source code processing They parse the program text to discover erroneous conditions Very effective as an aid to inspections; supplement but cannot replace inspections Particularly valuable for languages such as C that have weak typing (many errors can remain undetected by the compiler) Less cost-effective for languages such as Java that have strong type checking (many errors can be detected during compilation)

. Automated Static Analysis… Automatic static analysis checks [Fig. 19. 8, Somm 00] 25

. . Automated Static Analysis. . n Stages of static analysis: u u u 26 Control flow analysis. Checks for loops with multiple exit or entry points, finds unreachable code, etc. Data use analysis. Detects un-initialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc. Interface analysis. Checks the consistency of routine and procedure declarations and their use

…Automated Static Analysis. n Stages of static analysis [cont’d]: u u 27 Information flow analysis. Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code review (inspection) Path analysis. Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process

138% more lint_ex. c LINT static analysis [Fig 19. 9, Somm 00] #include <stdio. h> printarray (Anarray) int Anarray; { printf(“%d”, Anarray); } main () { int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ; } 139% cc lint_ex. c 140% lint_ex. c 28 lint_ex. c(10): warning: c may be used before set lint_ex. c(10): warning: i may be used before set printarray: variable # of args. lint_ex. c(4) : : lint_ex. c(10) printarray, arg. 1 used inconsistently lint_ex. c(4) : : lint_ex. c(11) printf returns value which is always ignored