Скачать презентацию Automated Software Testing Test Execution and Review Amritha Скачать презентацию Automated Software Testing Test Execution and Review Amritha

7b1792dc5b371edf0f0a67a6fa91796e.ppt

  • Количество слайдов: 19

Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm 16 u) Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm 16 u)

Contents • What is test execution? • Phases of Testing – Unit Testing – Contents • What is test execution? • Phases of Testing – Unit Testing – Integration Testing – System Testing • Test Results Analysis of Regression Tests • User Acceptance Testing • Defect Tracking and Automated Tracking tool • Test Program Status Tracking and EVMS • Test Metrics Collection and Analysis – White Box – Black Box • Test Execution Review

Test Execution Flow Test Execution Flow

What Is Test Execution? • Stage 5 of the ATLM • Creation of testing What Is Test Execution? • Stage 5 of the ATLM • Creation of testing environment • Tests are carried out on the application – Input = test procedures – Output = satisfied/altered acceptance criteria • Software Problem Reporting • E. g. Course. Marker

Phases of Testing • Unit, Integration, System and User Acceptance testing. • Tests executed Phases of Testing • Unit, Integration, System and User Acceptance testing. • Tests executed according to the Test Plan for each phase.

Phases of Testing: Unit Testing • Consistent with Development Schedule • Automated Results Checkout Phases of Testing: Unit Testing • Consistent with Development Schedule • Automated Results Checkout Process • White Box Level – Involves detailed look at code • Best to execute unit tests as soon after the code has been created. • Static Analysis • Code Profiling • Development Test Tools • Test Documentation – Metric Collection – Process Improvement – Software development folders (SDF) and Unit Test Evaluation Criteria

Phases of Testing: Integration Testing • “Units that are incrementally integrated are tested together Phases of Testing: Integration Testing • “Units that are incrementally integrated are tested together based on control flow” • Some integration testing takes place during unit testing • Test procedures based on integration test design addressed earlier in the ATLM • Results are analyzed and documented • Refining Test Procedures • Some test scripts can be automated for re-use in system testing • End-user approval = end of unit and integration testing

Phases of Testing: System Testing • • Higher Level Performed by a separate test Phases of Testing: System Testing • • Higher Level Performed by a separate test team May require many individual test procedures Test Execution – False Negatives – False Positives • Software Problem Report

Test Results/Analysis of Regression Tests • New application baseline – Build release notes should Test Results/Analysis of Regression Tests • New application baseline – Build release notes should accompany the new build • Smoke Tests • Regression Testing (after defects have been fixed/modified) – Investigate high risk areas of code – Indicator of developer behaviour • Test team investigates areas with larger number of problem reports • Analysis is a clear indicator of whether test execution procedures are useful in locating errors. – Where more test effort is required • System testing is complete when criteria have been met

User Acceptance Testing • • Involves End-user participation Consists of the subset of the User Acceptance Testing • • Involves End-user participation Consists of the subset of the suite of tests performed at system level Defects at UAT level will be documented in SPRs Satisfactory resolutions of level 1 and 2 SPRs = conclusion to testing

Defect Tracking and New Build Process • • Test team performs problem reporting operations Defect Tracking and New Build Process • • Test team performs problem reporting operations in compliance with a defined process – SPR creation – E-mail Notification to configuration management(CM) and developers New software is checked via CM tools New software build issuance Defect Priority – Fatal, high, medium, low Engineering review Board (ERB) Automated Defect Tracking Tool – Ensures reported defects receive proper attention – Central defect tracking repository – Has several basic characteristics Automated Test Tools – Maintain test results/permit automatic generation of test results Recommended that SPRs are generated per life-cycle stage

Test Program Status Tracking and EVMS • Helps to efficiently handle problems that arise Test Program Status Tracking and EVMS • Helps to efficiently handle problems that arise during test effort • Test engineer must produce meaningful reports based on test plan measures and metrics – Test log – Test coverage reports

Test Program Status Tracking and EVMS Cont… • • To effectively monitor testing progress Test Program Status Tracking and EVMS Cont… • • To effectively monitor testing progress and report to senior management, an earned value approach is used. EVMS = one of the best ways to track progress Tracks value of completed works and compares it to planned costs and actual – Provides true measure of schedule and cost status – Enables creation of effective corrective actions 4 steps – Identify short tasks – Schedule each task – Assign a budget to each task – Measure progress of each task Earned value for work completed – planned budget = Schedule Variance Earned value for work completed – actual cost = Cost Variance

Test Metrics Collection and Analysis • Metrics = key indicators of test coverage progress Test Metrics Collection and Analysis • Metrics = key indicators of test coverage progress and quality of test effort • Separated into white box and black box testing efforts • HOWEVER, gathering too many metrics can be time consuming

Test Metrics Collection and Analysis: White Box Testing • Targets application’s internal working • Test Metrics Collection and Analysis: White Box Testing • Targets application’s internal working • Measures the DEPTH of testing – Coverage Analysis • Code Analysis and Code Profiling e. g. Rational Quantify – Objective is to locate complex areas of source code • Fault Density (Fd) = Nd/KSLOC – Predicts the remaining number of faults • Design Complexity Metrics measures the number of ways a module can cal other modules

Test Metrics Collection and Analysis: Black Box Testing • Focus on the BREADTH of Test Metrics Collection and Analysis: Black Box Testing • Focus on the BREADTH of testing • Based on application’s external considerations • Each metric is assigned to one of three categories – Coverage – Progress – Quality

 • • • Test Metrics Collection and Analysis: Black Box Testing Coverage – • • • Test Metrics Collection and Analysis: Black Box Testing Coverage – Test Coverage – System Coverage – Functional Coverage Progress – Test Procedure Execution Status – Error Discovery Rate – Defect Aging – Defect Fix Rate – Defect Trend Analysis Quality – Test Success Index – Defect Density – Test Effectiveness – Problem Report Acceptance Criteria Metric – Test Automation Metric

Test Execution Review • Stage 6 of ATLM • Review test performance e. g. Test Execution Review • Stage 6 of ATLM • Review test performance e. g. metrics – Helps determine enhancements to be made • Document well performed activities • Lessons Learned should be acknowledged/documented – Aim to ensure best practices are used • Integrate a continual iterative learning process – Encouraged as improvement opportunities • “check pulse” • Documentation should be easily accessible to all team members • Corrective actions suggested must be evaluated • Represents he need for process improvement

Questions? Questions?