Agile Software Development and ISO IEC Software Quality Standards

Agile Software Development and ISO/IEC Software Quality Standards: Measuring Economic Benefits and Calculating Quantitative Yields Anca-Juliana Stoica, Margus Nael Royal Institute of Technology, KTH-ITC, Sweden 25 th International Forum on Systems, Software, and COCOMO Cost Modeling University of Southern California, Los Angeles, CA November 2 -5, 2010

Outline – Research Background • Agility vs. Discipline in Software Development • ISO/IEC Software Quality Standards – – – Research Objectives Research Design Case Studies • • • – – Techniques and Features Used Framework for assessing a project Metrics and Characteristics Quality Metrics Integrated Models for Measuring Economic Benefits and Calculating Quantitative Yields Results Conclusions

The Spectrum of Software Engineering Methods

Examples of Software Engineering Methods • Agile methods: – – Increased customer satisfaction Lower defect rates Faster development times Solution to rapidly changing requirements Examples: XP(Extreme Programming), Cristal Methods, ASD(Adaptive Software Development), DSDM, FDD, Scrum • Plan-driven methods: – Predictability – Stability – High assurance Examples: Cleanroom, PSP, TSP, CMM methods (Capability Maturity Models: Software CMM, Integrated CMM(CMMI)), ISO/IEC Software Quality Standards (9001 and the recent SQua. RE standards)

Agility vs Discipline

Agility vs. Discipline

Challenge Balance the two approaches to take advantage of their strengths in a given situation while compensating for their weaknesses

Two suggested solutions • • Software risk management approach Software quality management approach, for structuring projects to incorporate both agile and plan-driven features in proportion to project’s needs.

Trends for the 21 st Century Software Systems • For large – to very large scale software intensive systems, the Emerging Scalable Spiral process model • For small – to medium scale software systems, hybrid methods, using some other concepts in addition to software risk management, like: software quality management. Our focus here is on small – to medium scale software systems that need to balance agility and quality assurance.

ISO/IEC SQua. RE Series – Software product Quality Requirements and Evaluation – SQua. RE series of International Standards is based on 2 predecessors: • • ISO/IEC 9126 (Software product quality) ISO/IEC 14598 (Software product evaluation) – New general reference model consisting of 5 quality divisions: • • • Quality Management Division Quality Model Division Quality Measurement Division Quality Requirements Division Quality Evaluation Division.

Organisation of SQua. RE series of standards

SQua. RE Common Models – SQua. RE general reference model - navigation guide as a function of user’s task(s) – Software product LC model – views of internal, external software quality and software quality in use during software LC – Quality model structure –categorisation of software quality attributes into characteristics, sub-characteristics, and quality attributes. Remarks – Measuring and evaluating characteristics will assure quality – Organization must support the project with necessary equipment and knowledge – Create Quality Evaluation Project Plan

ISO/IEC SQua. RE Software Product Quality Views

Research Objectives – Study if there any conflicts between the application of agility principles and the ISO/IEC SQua. RE series of software quality standards – Guidelines on how to apply SQua. RE to agile (hybrid) software development – Real-life case studies for proving major benefits: • Software quality • Economic benefits.

Research Design – Data collection • • – Guidelines for applying ISO/IEC SQua. RE • • • – Principles How they apply to a specific project/development environment Use of inductive approach Real-life case studies with/without using SQua. RE • • – Software project assessment framework based on agile/plan -driven characteristics Experience base Software quality improvements Economic benefits and other quantitative yields Data analysis

Real-Life Case Studies Focus on two separate but similar real-life software development projects: 1. Project A -project without ISO/IEC SQua. RE 2. Project B -project with ISO/IEC SQua. RE Applied comparative analysis of projects and software metrics

Real-Life Case Studies Projects’ characteristics: • Modern Web-based small- sized software business systems • Stakeholder committment based on Theory W principles • Incremental development and delivery • Balancing agility and discipline • Software quality management done through the whole life-cycle • Use of a tailored software engineering method.

Project. A/Proje ct B • To publish fresh publications, e. g. news articles from news providers, via WAP or GPRS to the customers of a tecommunication company • Publications are delivered to application by RSS and server creates pages that are suitable for mobile stations to read. • To publish Web Logs (Blogs) via WAP or GPRS to the custommers of a telecommunication company • Publications are delivered to application by RSS and server creates pages that are suitable for mobile stations to read.

Company vision – – by – – capture a larger market share become more attractive to customers developing better software quality products managing better software development processes.

Project A Used techniques: – – – Feature Driven Development Small team Osmotic communication Spiking, Walking skeleton Information radiators Simple design Access to expert user Estimation using comparison and analogy Frequent delivery Documentation is not so important Kick-off meeting Status reporting

Project A Home ground polar chart

Project A Framework for project assessment Included in the experience base

Project B Used techniques: – – – Feature Driven Development Small team Osmotic communication Spiking, Walking skeleton Information radiators (cont. )

Project B Used techniques: – – – – – Simple design Acess to expert user Estimation using comparison and analogy Frequent delivery Kick-off meeting Status reporting Code reviews Measurements of quality characteristics Burn charts

Project B Home ground polar chart

Project B Framework for project assessment:

Project A Quality Model Dark colors mm represent quality n characteristics of the project

Project B Quality Model Remark: We have two quality models, Project A and Project B. We just have to find a way to measure it and prove that Project B is better.

Metrics and Characteristics – – – Project metrics: Personnel. Product metrics: Lifetime, Downtime, Number of defects, Complexity. Product quality metrics: Mean time to failure, Defect density, Customer satisfaction. Other metrics: Lost amount of money to customer, Lost amount of money to developer. Quantitative economic yields of improving software quality Remarks 1. Software quality metrics measured with Research Standard Metrics (RSM) tool 2. Quantitative economic yields of improving software quality are calculated (estimated) using a value-based framework.

RSM quality report

Project basic data (RSM Reports)

Software quality report (1): Total quality notices

Software quality report (2): Maintainability(Readability)

Software quality report (3): Quality notices

Software quality notices

Quantitative yields of improving software engineering methods Stoica, A. J. (2007). Exploring Quantitative Yields of Software Engineering Methods in the Software Life-Cycle. The first IEEE Computer Society EQUITY International Conference. 19 -21 March 2007, Amsterdam, the Netherlands.

Motivation The problem of exploring quantitative yields of a new (improved) software engineering method in the software life-cycle has to be addressed in a systemic manner – relevant metrics and models have to be selected – their interrelashionships have to be expressed in a formal manner to detect possible inconsistencies and/or incompletenesses. The Model Systems (MS) concept is applied to – analyze multiple facets of software development – derive a value-based framework (VF) for exploring the quantitative yields of a new (improved) software engineering method in the software life-cycle.

Value-Based Framework (VF) Interrelated software system metrics and models that address: – Costs (CM) – Quality (QM) – Productivity (WM) – Benefits (BM) from applying better software project quality management – Resulting value estimation metrics (VM), evaluated over the software life-cycle.

Quantitative Models for Estimating Costs, Benefits, and Value Applicable for software engineering methods that: – Improve software quality via structured and facilitated group reviews – Certify companies to adhere to regulations for international trade – Evaluate, qualify, certify suppliers to government agencies – Perform individual and group-based project management via technical challenging approaches – Use sets of criteria designed for supplier selection and establishment of priorities for internal software process improvement.

VF Instantiation Example • • Improving an agile (hybrid) software engineering method is achieved as a result of applying ISO/IEC software quality (SQua. RE) standards Metrics and models are applied to: i) existing, and ii) new (improved) software development process in order to calculate all the possible quantitative yields Improvements to an agile (hybrid) software engineering process are in similar to applying Software Inspection Method (SIM) Guidance of practical use that ISO/IEC SQua. RE quality standard contains is also quite similar.

Input parameters (Project B)

General metrics

Cost metrics

Benefit metrics

Value metrics

Main results – The real return on investment is 4700%. – The real gross benefit is 49701, 6 Eur. – Real net benefit when you apply Software inspection Method (SIM) 48666, 15 Eur. – Real benefit to cost ratio i. e. the ratio between benefits and costs is 48. – Real breakeven point is 24288, 90 Eur, or 202, 12 hours.

Summary – – – Our research shows that there are no conflicts between applying ISO/IEC SQua. RE software quality standards and agile software development Economic and quality benefits are demonstrated using a value framework as well as software metrics and measurements Simple guide to SQua. RE practical use with real-life example is presented Evaluation framework for software system projects to decide on the balance between agile and plan-driven approaches is outlined Experience base for analysing software system projects is created Our work is a step forward in reconciling agile approaches and formal software product quality evaluation that is discussed in ISO/IEC software quality standards.

Conclusions – We showed that creatively using software quality standards in agile software development is possible for • • • – – combining the best of agility and discipline creating quality software generating economic benefits. During the last years, agile software development has become a work facet in small and medium software development organizations, but its lack of theoretical foundations require practical methods, models, metrics, and/or processes to relate it to software engineering research and application as well as to the associated engineering standards. Our work supports this effort and offers theoretical and practical support for future advances in educational and software development organizations.