IBM Haifa Research Lab An Open Source Simulation Model of Software Testing Shmuel Ur Elad Yom-Tov Paul Wernick http: //qp. research. ibm. com/concurrency_testing IBM Haifa Labs
IBM Haifa Labs What Are the Development Tasks? } Writing Code n Creating lines of code, creating bugs } Testing n Creating tests, running tests } Reviews n Searching the code for bugs } Debugging n Looking for the bug root cause } Fixing n Replacing some code that has a bug o Putting new bugs in? } We will ignore specification, design, maintenance 2
IBM Haifa Labs Simulation Program Development } A program of certain size with certain quality need to be created n Size is measured in LOC n Quality is measured in # of bugs } A developer may spend time doing of the development tasks, The task may result in artifacts being created n Writing code create new line objects, one for each line, which may contain bugs n Writing tests, results in tests each of which can test a subset of the line objects. Automated tests are more expensive. n Running a test takes some time (depends if automated) and may find bugs in specific line objects. Running a test a second time is not as effective n… 3
IBM Haifa Labs Default Simulation Values } # Code lines per hour = 30 } Bugs per line = 0. 01 } Debugging time Correction = 1 + (LOC since bug)/2000 } Hours to fix bug = base 2, multiply by 2 if a month passed, multiply by 3 if two months passed } Probability discovered by inspection = 0. 5 : Laitenberger and De. Baud suggest that 70% is achievable } # Lines reviewed per hour = 200 } Cost of testing = 1. 14 hour per one hundred lines to do unit testing n Other measures exist, cost of creating tests, cost of running tests } Probability of finding bugs in test =. 5 n Probability of finding bug when re-running the test = 0 4
IBM Haifa Labs Simulation Cycle (a) Code building cycle : iterative Write code Test / Inspect (b) Code building cycle: Test First Write Test Write code Run Test Debug Check Debug Test /inspect Debug Check Quality Check Size Done 5 (c) System test Done
IBM Haifa Labs Sample Simulation 6
IBM Haifa Labs Sample Simulation 7
IBM Haifa Labs Overnight Regression – Chaotic behavior 8
IBM Haifa Labs Overall Productivity when Code Creation Rate Increase 9
IBM Haifa Labs What do Developers Really Do? 10
IBM Haifa Labs Impact of Length of Iteration in Iterative Software Development 11
IBM Haifa Labs Is Pair Programming a Good Idea? Depends!! 12
IBM Haifa Labs Automating Testing May be Overrated 13
IBM Haifa Labs Future Direction } Encapsulation n Reduce cost of debugging, review … } Code complexity } Planning tool n Quality requirement impact on methodology n Not one size fit all 14
Скачать презентацию IBM Haifa Research Lab An Open Source Simulation
0f9300459fd3157411c8fda44b2cc26a.ppt