Software Quality The Elusive Target Kitchenham B

Software Quality : The Elusive Target Kitchenham B. & Pfleeger S. L. January, 1996 IEEE Software Zhenyu Wu Software Quality CS 5391 Spring 2003 1

Introduction Questions: What do you really mean by software quality (SQ)? Is your definition adequate? How to evaluate your product’s SQ ? How to measure SQ? How to achieve SQ? Software Quality CS 5391 Spring 2003 2

Meaning of SQ A good definition should let us measure quality in a meaningful way Measurements let us know l l if techniques improve quality how process quality affects product quality how quality methods affect product quality during use Is the investment in SQ methods profitable? Software Quality CS 5391 Spring 2003 3

Meaning of SQ… Depends on how you approach software quality Different groups involved have different views l Product-based l Process-based People believe quality is important and can be improved Software Quality CS 5391 Spring 2003 4

Views of SQ SQ can be perceived in various domains (philosophy, economics, marketing and operations management) SQ is complex and multifaceted concept and can be described from five different perspectives l Transcendental view l User view l Manufacturing view l Product view l Value-based view Software Quality CS 5391 Spring 2003 5

Transcendental View See quality as something that can be recognized but not defined Quality as something toward which we strive as an ideal, but may never implement completely Abstract sense Software Quality CS 5391 Spring 2003 6

User View More concrete, grounded in the product characteristics that meet user’s needs Evaluate product in a task context can be personalized view Asses product behavior with respect to operational profiles Related to product usability and reliability Software Quality CS 5391 Spring 2003 7

Manufacturing View Focus on product quality during production and after delivery Concerns whether or not the product was constructed “right the first time” Can lead to quality assessment independent of the product Adapted by ISO 9001 and CMM, advocates conformance to process rather than to specification Software Quality CS 5391 Spring 2003 8

Product View Consider the product’s inherent characteristics, looks inside Assumption : measuring and controlling internal product properties will result in improved external product behavior More research needed to conform this assumption Software Quality CS 5391 Spring 2003 9

Value-Based View Different views can be held by different groups involved in software development l l user’s requirement for a useful product conflict with manufacturer’s goal of minimizing rework consider the trade-offs between cost and quality manage conflict design to cost compare with potential benefits This view tries to bring in compromise between different groups and views Software Quality CS 5391 Spring 2003 10

Measuring Quality We need to measure quality so we can: l l l Establish baselines Predict likely quality Monitor improvement Product attributes contribute to user satisfaction are a mixture of: l l l product’s functions product’s nonfunctional qualities constraints ISO definition of quality: the totality of characteristics of an entity that bear on its ability to satifsfy stated and implied needs. Software Quality CS 5391 Spring 2003 11

Measuring user's view Reliability: how long the product functions properly between failures Usability: including ease of installation, learning and use Tom Gilb’s technique: characteristics can be measured directly l Example: Learning time = average elapsed time (in hours) for a typical user to achieve stated level of competence Software Quality CS 5391 Spring 2003 12

Measuring user's view… This technique can be generialized to any quality feature The quality concept is broken down into component parts until each can be stated in terms of directly measurable attributes Software Quality CS 5391 Spring 2003 13

Measuring the Manufacturer's View Defects count Count the number of known defects recorded during development and use l Can compare modules, products or projects (with same way and same time) l Relationship between defects count and operational failures is unclear l Can indicate test efficiency and identify process improvement areas l Software Quality CS 5391 Spring 2003 14

Measuring the Manufacturer's View… Rework costs l l l Any additional effort required to find and fix problems after documents and code are formally signed-off as part of configuration management Count debugging effort during integration and testing Not count end-phase verification and validation Pre-release rework: measure manufacturing efficient and process improvement Post-release rework: measure of deliverd quality Software Quality CS 5391 Spring 2003 15

Modeling Quality Several models have been built to understand measure quality Old Models Mc. Call’s model l ISO 9126 l New model l Dromey’s model Software Quality CS 5391 Spring 2003 16

Mc. Call's Model Defines product qualities as hierarchy of factors, criteria and metrics: Quality factor represents behavior characteristic of the system l Quality criterion is an attribute of a quality factor that is related to software production and design l Quality metric is a measure that captures some aspect of a quality criterion. l Software Quality CS 5391 Spring 2003 17

Mc. Call's quality Model… The metrics have to be answered with ‘yes’ or ‘no’ answer The final quality is assessed by dividing the ‘yes’ answers by the total number of questions chosen to describe the quality Software Quality CS 5391 Spring 2003 18

Mc. Call’s Model… Software Quality CS 5391 Spring 2003 19

ISO 9126 Six characteristics, completely hierarchical Quality charactertistics are divided into sub-characteristics which can be measured using metrics Recommends measuring the characteristics directly, but does not indicate clearly how to do it Software Quality CS 5391 Spring 2003 20

Quality Characteristic Definition Functionality A set of attributes that bear on the existence of a set of functions and their specified propertise. The functions are those that satisfy stated or implied needs. Relibity A set of attributes that bear on the capability of software to maintain its performance level under stated conditions for a stated period of time. Usablity A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users. Software Quality CS 5391 Spring 2003 21

Efficiency A set of attributes that bear on the relationship between the software’s performance and the amount of resources used under stated conditions. Maintainability A set of attributes that bear on the effect needed to make specified modifications (which may include correction, improvements, or adaptations of software to environmental changes and changes in the requirements and functonal specifications). Portability Software Quality A set of attributes that bear on the ability of software to be transferred from one environment to another (this includes the organizational, hardware or software environment). CS 5391 Spring 2003 22

ISO 9126… Functionality Suitability Accurecy Interoperability Secuity Reliebility Materity Fault Telerance Recoverability Usability Understandability Learnability Operability Characteristics Software Quality Sub-characteristics CS 5391 Spring 2003 23

ISO 9126… Effiency Time behavior Resource behavior Maintainability Analyzability Changeability Stability Testability Portability Adaptability Installability Conformance Replaceability Characteristics Sub-characteristics Software Quality CS 5391 Spring 2003 24

Main Differences between Mccall’s and ISO 9126 model Mccall’s ISO 9126 with different quality frame work and terminology quality factor not hierarchical Software Quality quality characteristic and subcharacteristics completely hierarchical CS 5391 Spring 2003 25

Model Problems Lack rationale for determining which factor should be included in the quality definition Lack rationale for deciding which criteria relate to a particular factor The measure of metrics can be too subjective Does not describe how lower level metrics are composed into an overall assessment of higher level quality characteristics Software Quality CS 5391 Spring 2003 26

Dromey's models (new model) Higher level quality attributes cannot be measured or built directly l Higher level quality can be achieved by building components which exhibit a consistent, harmonious and complete set of product properties that result in manifestations of quality attributes l Can allow us to verify models l Software Quality CS 5391 Spring 2003 27

Business Value of Quality Businesses invest valuable money for obtaining good software, we have to pay them back in good We should be careful in assessing and providing software with good quality How much "less than perfect" software the businesses willing to accept should be determined Software Quality CS 5391 Spring 2003 28

Conclusion Quality is a complex concept; It means different things to different people You must define aspects of quality in which you are interested Quality should be defined in a measurable way, so that it can be understood and verified Quality must be related to business goals Software Quality CS 5391 Spring 2003 29