Software Quality Assurance Design and implementation
Objectives • At the end of this lecture you should be able to; – list and describe factors which contribute to software quality – name and explain the processes used to ensure software quality COMP 319 S Coope 2011 slide 2
Quality “Characteristics of the software and/or its supporting documentation which make it suitable for its purpose". COMP 319 S Coope 2011 slide 3
Product design and conformance • Design – characteristics of the materials and processes which have been specified for the system – includes all aspects of specification and software design • Conformance – degree to which design specifications are followed during manufacturing – code implementation COMP 319 S Coope 2011 slide 4
De. Marco’s Quality “A products quality is a function of how much it changes the world for the better” COMP 319 S Coope 2011 slide 5
Quality control Production design process 1 Production design process 2 Review Sample Test Measure COMP 319 S Coope 2011 slide 6
Software production quality cost • Each individual product needs producing only once • Benefits – economy of scale (sometimes) • Drawbacks – all products from one release will contain same defects – implementation details may change radically from product to product – statistical sampling techniques not applicable COMP 319 S Coope 2011 slide 7
Software verses other manufacturing • Manufacturing industry – design once, make many – production process refined – design faults fed back on-going basis • Software industry – design once, make once COMP 319 S Coope 2011 slide 8
Component quality concept • In manufacturing – electronics, cars, aviation – components made to specification and determined to be of high quality – build product out of blackbox components • In software – components conform to interface specification – MS Component Object Model – Sun Java Beans COMP 319 S Coope 2011 slide 9
Quality and reuse High reuse should improve quality as code needs to only be tested or designed once only COMP 319 S Coope 2011 slide 10
Need for quality control Cost of correcting errors • IBM research – 7053 hours inspecting 200, 000 lines of code, 3112 potential defects discovered – total cost $282, 000 – cost per defect (at $40/hour) $91/defect – IBM estimate for correcting defects in the field is $25, 000 – total cost of 3112 defects $5 million COMP 319 S Coope 2011 slide 11
Reliability Estimation • Calculating probability of fault over time • Why do we need it? – schedule maintainance – commercial (costing for warrantees, support contracts) – scheduling software release dates – determining safety of critical systems COMP 319 S Coope 2011 slide 12
Reliability Estimation COMP 319 S Coope 2011 slide 13
Reliability Problems • Therac 25 • Airbus 320 – altimeter reading issues • Patriot defence missile failure – Scud missile killed 28 – 0. 000000095/0. 1 second over 100 hours COMP 319 S Coope 2011 slide 14
Reliability modelling • Defect density models – analyses structure/size of software – input/outputs/complexity – can be determined before execution • Reliability growth models – test performance – execution time – correlates defect density over time COMP 319 S Coope 2011 slide 15
Reliability growth model • Parameters – Total bugs in software – Total bugs discovered – Time – Residual bugs COMP 319 S Coope 2011 slide 16
Reliability growth model • Exponential COMP 319 S Coope 2011 slide 17
Growth model types COMP 319 S Coope 2011 slide 18
Growth model examples A = a(1 -e-bt) – Goel-Okumoto or Musa model – Concave a=total defects A = a( 1 -(1+bt)e-bt) – Yamada, 83 Modification of G-O model • A= (l/c)ln(c t+l) – Infinite poisson Musa COMP 319 S Coope 2011 slide 19
Reliability growth model assumptions • Defects are repaired immediately – Defects are not repaired immediately • Defect repair doesn't introduce new defects – Generally not true • Test cases represent operation profile of software – hard to duplicate in practise, need for alpha release phase • Failures are independent COMP 319 S Coope 2011 slide 20
Using a model • Parameter estimation – maximum likelihood – least squares • Maximum likelihood – Given a set of data – Which distribution gives us the highest probability of achieving this data COMP 319 S Coope 2011 slide 21
Maximum liklihood example • A coin is tossed 10 times • Chance of getting N heads is – (10!/(N!(10 -N)!) x (phead)^N x (1 -phead)^(10 -N) • So given we get 3 heads – P=(10!/(3!(7)!) x (phead)^3 x (1 -phead)^7 • What is the chance of – phead being 0. 1, 0. 2, 0. 3 etc. • We can define these as – P(3 | 0. 1) P(3 | O. 2) P(3 | 0. 3) or generally – P(3 | p) COMP 319 S Coope 2011 slide 22
Maximum liklihood example • Now imagine we have the following data (number of heads) – 2, 2, 2, 3, 3, 5 • We end up with the following data – P(2 | p) P(3 | p) P(5 | p) • We can multiply all those probabilities together (since they are independent) – P(2 | p) x P(3 | p) x P(5 | p) • We then solve for max prob. Against p COMP 319 S Coope 2011 slide 23
Least Squares (regression) • In this approach the parameters are chosen which minimise the error squares • (F(a, b, t)-f(t))^2 COMP 319 S Coope 2011 slide 24
Useful estimation • Stable – not vary widely in terms of total bugs estimated week per week • Give a good indication of residual errors after release – release conditions can be very different than test conditions COMP 319 S Coope 2011 slide 25
Case study (Tandem) • Tandem – fault tolerant computer systems (now part of HP) • Results – Release Predicated faults Faults year 1 – 1 33 34 – 1/2 33 28 – 3 10 9 COMP 319 S Coope 2011 slide 26
Tandem conclusions • Simplest models are the best • Execution time is the only time measure worth using • Reliability growth is a useful measure • Problem reports are a good substitute for actual defects counts (problem reports can include duplicates) • Data can be analysed on a weekly basis COMP 319 S Coope 2011 slide 27
Mc. Call's software quality factors COMP 319 S Coope 2011 slide 28
Product operations • Correctness – conformance to specification (verification) – conformance to user requirements (validation) – conformance to module specification • Reliability – results produced to expected precision in all circumstances • Efficiency – efficient use of hardware resource COMP 319 S Coope 2011 slide 29
Product operations • Integrity – keeps data secure – commonly conflicts with functionality requirements • Useability – easy to use – intuitive interface COMP 319 S Coope 2011 slide 30
Product revision • Maintainability – can the software be fixed at a later date • Flexibility – can the software be modified to perform new or different functions • Testability – can the software be tested COMP 319 S Coope 2011 slide 31
Product transition • Portability – can the software be used on another hardware or software platform • Reusability – can the software be used to produce new software • Interoperability – will we be able to interface to another system COMP 319 S Coope 2011 slide 32
Evaluating and assuring quality • Product operations – testing (user, system, security, module, stress, usability) • Possible metrics – faults per line of code/module – mean time between software failure – time to perform operations (usability) – mean time between break ins (security) – mean response time under stress COMP 319 S Coope 2011 slide 33
Product revision quality control • Maintainability – review (documentation, code complexity) • Flexibility – where to put parameters – review internationalisation issues • Testability – reduce complexity – encapsulate data (OO paradigm) COMP 319 S Coope 2011 slide 34
Flexibility configuration parameters • Hardcoding – very bad practise • Using #defines or constants – better, good for system constants – more control than configuration file – requires recompilation • Configuration file – highest flexibility – allows users to break software !! COMP 319 S Coope 2011 slide 35
![Examples Example 1 Hard coding Account accounts[]=new Account[20]; for (j=0; j<20; j++) { accounts[j]=file.](https://present5.com/presentation/1af098ea2dad1e7e5256755842f20e99/image-36.jpg "Examples Example 1 Hard coding Account accounts[]=new Account[20]; for (j=0; j<20; j++) { accounts[j]=file.")
Examples Example 1 Hard coding Account accounts[]=new Account[20]; for (j=0; j<20; j++) { accounts[j]=file. read. Account(); } Example 2 pre-processor definition #define MAX_ACCOUNTS 20 Account accounts[]=new Account[MAX_ACCOUNTS]; for (j=0; j<MAX_ACCOUNTS; j++) { accounts[j]=file. read. Account(); } COMP 319 S Coope 2011 slide 36
Example (configuration file) Example 3 configuration file int max_accounts; read. Config(&max_accounts, "config. dat", "maxacco unts"); Account accounts[]=new Account[max_accounts] for (j=0; j<max_accounts; j++) { accounts[j]=file. read. Account(); } COMP 319 S Coope 2011 slide 37
Transition • Portability – effected by language, software library support – note C does not have a standard size for int • Reusability – code should be generalised as much as possible (including variable naming) – e. g. replace pin with authorisation_code – important to de-couple I/O from processing COMP 319 S Coope 2011 slide 38
Portability • Approaches – use wrapper classes – define UI using XML – use plain old Java classes – use model/view/controller seperation COMP 319 S Coope 2011 slide 39
Transition • Interoperability – use standards e. g. Internet standards, JDBC, HTTP, XML – test against standard products e. g. browsers – next use in-house protocols if standard technique available COMP 319 S Coope 2011 slide 40
Software reviews • Review functionality of system – by development team – specification team – with customer (validation) – beta release to customer COMP 319 S Coope 2011 slide 41
FTRs Formal technical reviews • Purpose – uncover defects (code, spec, design) • Before – agenda, attendance list, documentation • While – walkthrough, code inspection • After – hit list, next review meeting scheduled COMP 319 S Coope 2011 slide 42
Specification review • Is it – clear and unambiguous – understandable – free of errors – complete – what the customer wants COMP 319 S Coope 2011 slide 43
Design review • Check – is it broken up enough – in agreement with specification – uses correct notation – makes sense COMP 319 S Coope 2011 slide 44
Code review • Is it – bug free – well document and formatted – uses meaningful names – flexible (easy to modify) – easy to understand/test COMP 319 S Coope 2011 slide 45
Defect amplification (importance of review) • Error in specification – errors in design – errors in code – errors in product • Error in design – errors in code – errors in product COMP 319 S Coope 2011 slide 46
Quality factors (environment) • • Staff turnover Training Size of organisation Resources (time and money) COMP 319 S Coope 2011 slide 47
Скачать презентацию Software Quality Assurance Design and implementation Objectives
1af098ea2dad1e7e5256755842f20e99.ppt