Скачать презентацию Advanced Software Engineering Software Testing COMP 3702 Instructor Скачать презентацию Advanced Software Engineering Software Testing COMP 3702 Instructor

921f537b4240142911b3b30395a94154.ppt

  • Количество слайдов: 49

Advanced Software Engineering: Software Testing COMP 3702 Instructor: Anneliese Andrews Thomas Thelin – Software Advanced Software Engineering: Software Testing COMP 3702 Instructor: Anneliese Andrews Thomas Thelin – Software Testing'05

Advanced Software Engineering: Software Testing COMP 3702 Instructor: Anneliese Andrews Thomas Thelin – Software Advanced Software Engineering: Software Testing COMP 3702 Instructor: Anneliese Andrews Thomas Thelin – Software Testing'05

News & Project ¨News ¨ Guest lecture 10/5 by Tomas Berling ¨Project ¨ Deadline News & Project ¨News ¨ Guest lecture 10/5 by Tomas Berling ¨Project ¨ Deadline 10/5: Report to supervisor and peer review group ¨ Project presentations 17/5, 8 -12 http: //serg. telecom. lth. se/education/Sw. Test/ Thomas Thelin – Software Testing'05

Lecture ¨Chapter 9 ¨ Metrics ¨Chapter 16 (& Appendix III) ¨ Process improvement and Lecture ¨Chapter 9 ¨ Metrics ¨Chapter 16 (& Appendix III) ¨ Process improvement and assessment ¨ TMM, TIM ¨Presentation technique Thomas Thelin – Software Testing'05

Test management questions ¨What is effective? ¨What is efficient? ¨When to stop testing? ? Test management questions ¨What is effective? ¨What is efficient? ¨When to stop testing? ? Thomas Thelin – Software Testing'05

Monitoring Thomas Thelin – Software Testing'05 Monitoring Thomas Thelin – Software Testing'05

Why measure? ¨Why is metrics important for the development and test process? ¨Are metrics Why measure? ¨Why is metrics important for the development and test process? ¨Are metrics useful for motivation purposes? ¨Can metrics improve quality? Thomas Thelin – Software Testing'05

Measurement basics Basic data: Basic rule: ¨Time (calendar and staff hours) ¨Feedback to origin Measurement basics Basic data: Basic rule: ¨Time (calendar and staff hours) ¨Feedback to origin ¨Failures / Faults ¨Use data or don’t measure ¨Size Thomas Thelin – Software Testing'05

Purpose of metrics ¨Project monitoring – check the status ¨Project controlling – corrective actions Purpose of metrics ¨Project monitoring – check the status ¨Project controlling – corrective actions ¨Plan new projects ¨Measure and analyze results ¨ The profit of testing ¨ The cost of testing ¨ The quality of the test process ¨ Basis of improvement, not only for the test process Thomas Thelin – Software Testing'05

Monitoring testing ¨Status ¨ Coverage metrics ¨ Test case metrics Metrics ¨Failure / Fault Monitoring testing ¨Status ¨ Coverage metrics ¨ Test case metrics Metrics ¨Failure / Fault metrics ¨ How much have we accomplished? ¨ What is the quality status of the software? Estimation ¨Productivity metrics ¨ How productive and efficient are the testers? ¨Efficiency / Cost metrics ¨ How much time have we spent? ¨ How much money have we spent? ¨Effectiveness metrics ¨ How effective is the testing techniques in detecting defects? Thomas Thelin – Software Testing'05 Cost Stop?

Test metrics: Size/complexity/length What? ¨Size – LOC, Function Points ¨Complexity – Mc. Cabe ¨Length Test metrics: Size/complexity/length What? ¨Size – LOC, Function Points ¨Complexity – Mc. Cabe ¨Length – Halstead Why? ¨Estimate test effort Thomas Thelin – Software Testing'05

Test metrics: Efficiency What? ¨# faults/hour ¨# faults/test case Why? ¨Evaluate efficiency of V&V Test metrics: Efficiency What? ¨# faults/hour ¨# faults/test case Why? ¨Evaluate efficiency of V&V activities Thomas Thelin – Software Testing'05

Test metrics: Effectiveness What? ¨% found faults per phase ¨% missed faults Why? ¨Evaluate Test metrics: Effectiveness What? ¨% found faults per phase ¨% missed faults Why? ¨Evaluate effectiveness of V&V activities Thomas Thelin – Software Testing'05

Test metrics: Coverage What? ¨% statements covered ¨% branches covered ¨% data flow Why? Test metrics: Coverage What? ¨% statements covered ¨% branches covered ¨% data flow Why? ¨Track completeness of test ¨% requirements ¨% equivalence classes Thomas Thelin – Software Testing'05

Test metrics: Test execution status What? ¨# faults/hour ¨# executed tests ¨Requirements coverage Why? Test metrics: Test execution status What? ¨# faults/hour ¨# executed tests ¨Requirements coverage Why? ¨Track progress of test project ¨Decide stopping criteria Thomas Thelin – Software Testing'05

Test metrics: Trouble reports What? ¨# faults/size ¨repair time ¨root cause Why? ¨Monitor quality Test metrics: Trouble reports What? ¨# faults/size ¨repair time ¨root cause Why? ¨Monitor quality ¨Monitor efficiency ¨Improve Thomas Thelin – Software Testing'05

Selecting the right metrics ¨What is the purpose of the collected data? ¨ What Selecting the right metrics ¨What is the purpose of the collected data? ¨ What kinds of questions can they answer? ¨ Who will use the data? ¨ How is the data used? ¨When and who needs the data? ¨ Which forms and tools are used to collect the data? ¨ Who will collect them? ¨ Who will analyse the data? ¨ Who have access to the data? Thomas Thelin – Software Testing'05

Goal-Question-Metric Paradigm (GQM) ¨ Goals ¨ What is the organization trying to achieve? ¨ Goal-Question-Metric Paradigm (GQM) ¨ Goals ¨ What is the organization trying to achieve? ¨ The objective of process improvement is to satisfy these goals ¨ Questions about areas of uncertainty related to the goals ¨ You need process knowledge to derive the questions ¨ Metrics ¨ Measurements to be collected to answer the questions Thomas Thelin – Software Testing'05

GQM guidelines ¨ Analyze <object(s) of study> ¨ the detection of design faults using GQM guidelines ¨ Analyze ¨ the detection of design faults using inspection and testing ¨ for the purpose of ¨ evaluation ¨ With respect to their ¨ effectiveness and efficiency ¨ from the point of view of the ¨ managers ¨ in the context of ¨ developers, and in a real application domain Thomas Thelin – Software Testing'05

Test estimation ¨Guessing ¨Authorized deadlines ¨Previous test effort ¨ In-project experience through incremental planning Test estimation ¨Guessing ¨Authorized deadlines ¨Previous test effort ¨ In-project experience through incremental planning ¨Formula-based ¨ COCOMO ¨ Test point analysis ¨Consensus of experts (Delphi) ¨Detailed work breakdown of testing tasks ¨Historical data helps estimating Software regardless of the method Thomas Thelin – Testing'05

Test Point Analysis Dynamic test points TPf=FPf*Df*Qd Static test points Total number of test Test Point Analysis Dynamic test points TPf=FPf*Df*Qd Static test points Total number of test points Environmental factors Skill factors Primary test hours PT=TP*S*E Total number of test hours Thomas Thelin – Software Testing'05 Planning and control factors

Cost of testing ¨How much does testing cost? ¨As much as resources we have! Cost of testing ¨How much does testing cost? ¨As much as resources we have! Thomas Thelin – Software Testing'05

Costs of testing ¨ Pre-run costs planning, specification, environment setup ¨ Execution costs execution Costs of testing ¨ Pre-run costs planning, specification, environment setup ¨ Execution costs execution time (staff and equipment), regression tests ¨ Post-run costs outcome analysis, documentation Thomas Thelin – Software Testing'05

Minimizing costs Test performance ¨ Automate execution and evaluation Test maintenance ¨ Add new, Minimizing costs Test performance ¨ Automate execution and evaluation Test maintenance ¨ Add new, review effectiveness Testware development ¨ Consider as part of the product Thomas Thelin – Software Testing'05

When to stop testing? ¨All coverage goals met (requirements, code, . . . ) When to stop testing? ¨All coverage goals met (requirements, code, . . . ) ¨Detection of specific number of failures ¨Rates of failure detection fallen below a specified level ¨Fault seeding ratios are favourable ¨Reliability above a certain value ¨Number of failures left below the limit ¨Cost has reach the limit – Software Thomas Thelin Testing'05

Most common stop criteria ¨Time has reach the limit (milestone, release date, . . Most common stop criteria ¨Time has reach the limit (milestone, release date, . . . ) ¨All planned tests executed and passed Thomas Thelin – Software Testing'05

Example Number of detected failures Number of executed test cases Number of failures per Example Number of detected failures Number of executed test cases Number of failures per day Interpretation? Thomas Thelin – Software Testing'05

Example Number of detected failures Number of executed test cases Interpretation? Thomas Thelin – Example Number of detected failures Number of executed test cases Interpretation? Thomas Thelin – Software Testing'05

Example Number of detected failures Number of executed test cases Interpretation? Thomas Thelin – Example Number of detected failures Number of executed test cases Interpretation? Thomas Thelin – Software Testing'05

Process models ¨Purpose ¨ Improvement ¨ Assessment ¨Models ¨ (Integrated) Capability maturity model (CMM, Process models ¨Purpose ¨ Improvement ¨ Assessment ¨Models ¨ (Integrated) Capability maturity model (CMM, CMMI) ¨ Software process improvement and capability determination (SPICE) ¨ ISO 9001, Bootstrap ¨ Test maturity model (TMM) ¨ Test process improvement model (TPI) Thomas Thelin ¨ Test improvement model (TIM) – Software Testing'05

CMM Thomas Thelin – Software Testing'05 CMM Thomas Thelin – Software Testing'05

Test Maturity Model (TMM) ¨Levels ¨Maturity goals and sub-goals ¨ Scope, boundaries, accomplishments ¨ Test Maturity Model (TMM) ¨Levels ¨Maturity goals and sub-goals ¨ Scope, boundaries, accomplishments ¨ Activities, tasks, responsibilities ¨Assessment model ¨ Maturity goals ¨ Assessment guidelines ¨ Assessment procedure Thomas Thelin – Software Testing'05

Mo d tu ri ty Ma Te st Thomas Thelin – Software Testing'05 Mo d tu ri ty Ma Te st Thomas Thelin – Software Testing'05

Change model Models Vision Strategies Processes Plans Organization Quality system Application Create Product Create Change model Models Vision Strategies Processes Plans Organization Quality system Application Create Product Create Ability Utilize Thomas Thelin – Software Testing'05 Architecture Design Software Hardware

Test improvement model – TIM ¨Key areas ¨Maturity levels ¨ Organization ¨ Initial ¨ Test improvement model – TIM ¨Key areas ¨Maturity levels ¨ Organization ¨ Initial ¨ Planning and ¨ Basic monitoring ¨ Cost effective ¨ Test cases ¨ Risk thinking ¨ Testware ¨ Optimizing ¨ Review / Inspection Thomas Thelin – Software Testing'05

Maturity profile (example) Thomas Thelin – Software Testing'05 Maturity profile (example) Thomas Thelin – Software Testing'05

Phases of assessment / improvement TMM TIM ¨ Plan (Preparation) ¨ Do (conducting, reporting) Phases of assessment / improvement TMM TIM ¨ Plan (Preparation) ¨ Do (conducting, reporting) ¨ Check (analyzing, action planning) ¨ Act (implementing improvement) 1. 2. 3. 4. 5. Evaluation – mirror of the test organization Present, discuss and reflect – understand the problems Develop improvement strategy Implement the improvement strategy Continually improvement Thomas Thelin – Software Testing'05

Evaluation ¨Interview based on a questionnaire ¨Covers the key areas and maturity levels ¨“Magic” Evaluation ¨Interview based on a questionnaire ¨Covers the key areas and maturity levels ¨“Magic” questions: ¨ What would you like to change? ¨ What would make your situation better? Thomas Thelin – Software Testing'05

Present, discuss and reflect ¨Write a report and present ¨All questions are discussed ¨Goal: Present, discuss and reflect ¨Write a report and present ¨All questions are discussed ¨Goal: create a common understanding Thomas Thelin – Software Testing'05

Develop improvement strategy ¨Prioritized areas are identified ¨Improvement puzzle ¨Support from the organization is Develop improvement strategy ¨Prioritized areas are identified ¨Improvement puzzle ¨Support from the organization is very important Thomas Thelin – Software Testing'05

Implement the improvement strategy ¨Seminars and workshops ¨Pilot project – internal mentors ¨Role play Implement the improvement strategy ¨Seminars and workshops ¨Pilot project – internal mentors ¨Role play Thomas Thelin – Software Testing'05

General advice ¨Identify the real problems before starting an improvement program ¨“What the customer General advice ¨Identify the real problems before starting an improvement program ¨“What the customer wants is not always what it needs” ¨Implement “easy changes” first ¨Involve people ¨Changes take time! Thomas Thelin – Software Testing'05

Presentation technique “Why on earth people who have something to say which is worth Presentation technique “Why on earth people who have something to say which is worth hearing should not take the slight trouble to learn how to make it heard is one of the strange mysteries of modern life. ” Sir Arthur Conan Doyle Thomas Thelin – Software Testing'05

uppmärksamhet Föredraget Minuter Thomas Thelin – Software Testing'05 uppmärksamhet Föredraget Minuter Thomas Thelin – Software Testing'05

Mitt sommarlov… Vi började med att Sedan följde. . Så småningom. . Till slut Mitt sommarlov… Vi började med att Sedan följde. . Så småningom. . Till slut kom vi fram till. . Star t Slut Thomas Thelin – Software Testing'05

Löpsedelstekniken - i vilken ordning vill lyssnaren veta? Star t Slut Thomas Thelin – Löpsedelstekniken - i vilken ordning vill lyssnaren veta? Star t Slut Thomas Thelin – Software Testing'05

ihågkomst Våra sinnen och minnet ord + bild ord Föredragets tidpunkt. . Ett antal ihågkomst Våra sinnen och minnet ord + bild ord Föredragets tidpunkt. . Ett antal dagar senare Thomas Thelin – Software Testing'05

Presentationsteknik ¨ Löpsedelstekniken ¨ Närhet ¨ Igenkänning ¨ Personifiering ¨ Konsekvenser Thomas Thelin – Presentationsteknik ¨ Löpsedelstekniken ¨ Närhet ¨ Igenkänning ¨ Personifiering ¨ Konsekvenser Thomas Thelin – Software Testing'05

This week ¨Project ¨ Write report ¨ Deadline 10/5: Report to supervisor and peer This week ¨Project ¨ Write report ¨ Deadline 10/5: Report to supervisor and peer review group ¨ Deadline 17/5: Comments (peer review) to supervisor and group ¨ Presentations 17/5, 8 -12. ¨Lab ¨ Process simulation ¨ Report of Lab 4 to Carina Thomas Thelin – Software Testing'05