ICS 52 Introduction to Software Engineering Lecture Notes

ICS 52: Introduction to Software Engineering Lecture Notes for Summer Quarter, 2003 Michele Rousseau Topic 14 Summer 2003 1

Process Improvement l l l Topic 14 Capability Maturity Model ISO 9000 PSP Summer 2003 2

Process improvement l l l Understanding existing processes Introducing process changes to achieve organisational objectives usually focused on: • quality improvement • cost reduction • schedule acceleration l Most work so far has focused on • defect reduction to improve Quality l Topic 14 Testing can’t do it all Summer 2003 3

Process improvement stages l Process analysis • Model and analyse (quantitatively if possible) existing processes l Improvement identification • Identify quality, cost or schedule bottlenecks l Process change introduction • Modify the process to remove identified bottlenecks l Process change training • Train staff involved in new process proposals l Topic 14 Change tuning Summer 2003 • Evolve and improve process improvements 4

The process improvement process Topic 14 Summer 2003 5

Process and product quality l Process quality product quality • These are closely related l l l Topic 14 A good process is usually required to produce a good product For manufactured goods, process is the principal quality determinant For design-based activity, other factors are also involved especially the capabilities of the designers Summer 2003 6

Principal product quality factors Topic 14 Summer 2003 7

What is CMM? l l l Capability Maturity Model Developed by the software community in 1986 with leadership from the SEI. Has become a de facto standard for assessing and improving processes related to software development Has evolved into a process maturity framework Provides guidance for measuring software process maturity Helps establish process improvement programs Topic 14 Summer 2003 8

What is the Software CMM? l l Topic 14 “A common-sense application of process management and quality improvement concepts to the software development and maintenance” A model for organizational improvement Summer 2003 9

Software Capability Maturity Model. Maturity Levels Process Capability Contain Indicate Key Process Areas Goals Organized by Achieve Common Features Implementation Address Contain Key Practices Activities Describe CMU/SEI-93 -TR-24 p. 29 Topic 14 Summer 2003 10

What makes up the CMM? l l The CMM is organized into five maturity levels: l Initial l Repeatable l Defined l Manageable l Optimizing Except for Level 1, each maturity level decomposes into several key process areas that indicate the areas an organization should focus on to improve its software process. Topic 14 Summer 2003 11

Each Maturity Level l 1. Initial : » ad hoc process. Success depends on individual effort. l 2. Repeatable : » Basic management processes: cost, schedule and functionality l 3. Defined : » Activities are documented, standardized and integrated into an organization-wide software process. l 4. Managed : » Detailed measures are collected: software and product quality. l 5. Optimizing : » Continuous process improvement: quantitative feedback from the process and from testing new ideas and technologies. Topic 14 Summer 2003 12

Five Levels of Software Process Maturity Continually improving process Predictable process Standard, consistent process Disciplined process Repeatable Initial Topic 14 Optimizing Managed Defined Focus on process improvement Process measured and controlled Process characterized, fairly well understood Level 5 Level 4 Level 3 Can repeat previously mastered tasks Unpredictable and poorly controlled Level 1 Level 2 Summer 2003 13

Key Process Areas • • KPAs associated with each maturity level describe functions that must be present to satisfy good practice at a particular level. (Except Level 1) Each KPA is described by: • Goals – Overall objectives • Commitments – Requirements that must be met to achieve the goals. • Abilities – Things that must be in place to enable the organization to meet the commitment. • Activities – Specific tasks required to achieve the KPA function • Methods for monitoring implementation • Methods for verifying implementation • Topic 14 Each KPA is defined by a set of key practices that contribute to satisfying its goals. (ie policies, procedures, and activities) Summer 2003 14

Level 2 KPAs - Purpose * Paraphrased Topic 14 Summer 2003 15

Level 2 KPAs - Purpose (Cont. ) Changes in commitments are agreed to by affected groups Topic 14 * Paraphrased All activities are planned and tracked Summer 2003 16

Level 3 KPAs - Purpose * Paraphrased Topic 14 Summer 2003 17

Level 3 KPAs - Purpose (Cont. ) * Paraphrased Topic 14 Summer 2003 18

Level 4 KPAs - Purpose * Paraphrased Topic 14 Summer 2003 19

Level 5 KPAs - Purpose * Paraphrased Topic 14 Summer 2003 20

Interesting CMM Facts · · · Topic 14 The number of companies using CMM to assess their software management practices more than doubles every five years (since 1987). Software Quality Assurance is the biggest obstacle for organizations trying to move from level 1 to level 2. Organization Process Definition is one of the biggest obstacles for organization trying to move from level 2 to level 3. Summer 2003 21

… more interesting facts · On average, it takes an organization: » 25 months to move from level 1 to 2 » 22 months to move from level 2 to 3 » 36. 5 months to move from level 3 to 4 · · Topic 14 About a third of companies engaged in CMM are located overseas (primarily India), and are 3 times more likely to reach CMM level 4 or 5 than US organizations. Only about 23% of organizations surveyed eventually move from level 2 to level 3 or higher. Summer 2003 22

What improved maturity provides Based on data from 1300 applications, average 200, 000 LOC Source: Master Systems, Inc. Topic 14 Summer 2003 23

Characteristics of Immaturity l l l Topic 14 Software process improvised during the course of a project. Even if process is specified, it is not rigorously followed or enforced. Reactionary, focus on solving immediate crises. Hard deadlines often mean a compromise in functionality and/or quality. No objective basis for judging product quality or for solving process problems. Quality is difficult if not impossible to predict. Summer 2003 24

Characteristics of Maturity l l l l Topic 14 Able to manage software development and maintenance organization/project wide. There is a prescribed, mandated, and enforced process. Process is consistent with the way that work actually gets done. Process is updated and improved as necessary. Roles and responsibilities within the process are clear. Quality is measured and monitored, and an objective basis for judgment exists. The necessary infrastructure for supporting the process exists. Workers see the value in the process. Summer 2003 25

Greater Maturity Can Bear Fruit l l l Topic 14 SE division of Hughes aircraft spent @$500 K over a three year period for assessment and improvement programs. By the end of the three year period, assessed at CMM Level 3. Estimated savings of @$2 M annually as a result (less overtime, less rework, greater productivity, etc. ) Equipment Division of Raytheon rise to CMM Level 3, at an estimated cost of @$580 K resulted in 2 -fold increase in productivity along with savings of @$15. 8 M in rework costs. Motorola GED (CMM Level 4) documented significant • reduction in cycle time • reduction in defect rates • increase in productivity 2003 Summer 26

ISO 9000 - Background l l l Topic 14 International set of standards for quality management (ISO 9000: 2000, ISO 9001: 2000, ISO 9004: 2000, etc. ) ISO is name adopted by the International Organization for Standardization (not an acronym) – comes from isos “equal” ISO 9000 is the most popular quality standard in the world Over 13, 000 standards issued since 1946 Made up of representative bodies from over 140 countries It applies to almost all types of organizations regardless of their function or product. Summer 2003 27

From “The Dilbert Principle” Topic 14 Summer 2003 28

ISO 9000 – What is it? l l l Topic 14 ISO 9000 is primarily concerned with "quality management". ISO 9001: 2000 specifies requirements for a quality management system for any organization that needs to demonstrate its ability to consistently provide product that meets customer and applicable regulatory requirements and aims to enhance customer satisfaction, in all business sectors It involves the development of a quality system that meets the quality requirements of the ISO standards. Summer 2003 29

Contents of Software Quality Plan from ISO 9000 Management responsibility l Quality system l Contract review l Design control l Quality control l Purchasing l Customer supplied info l Configuration management l Process control l Inspection and testing equipment Topic 14 l l l l l Control of non-conforming product Corrective action Handling, storage, packing and delivery Quality records Internal quality audits Training Software maintenance Statistical techniques Control of the development environment Summer 2003 30

ISO 9000 series of standards: l l l Topic 14 represents an international consensus on good management practices guidelines on what constitutes an effective quality management system serves as framework for continuous improvement Summer 2003 31

ISO 9000 and quality management Topic 14 Summer 2003 32

ISO 9000 - Certification l l l Topic 14 Quality standards and procedures should be documented in an organisational quality manual External body may certify that an organisation’s quality manual conforms to ISO 9000 standards (namely ISO 9001) Customers are, increasingly, demanding that suppliers are ISO 9000 certified Summer 2003 33

ISO vs CMM l l l Topic 14 CMM and the ISO 9000 series of standards share common concerns with quality and process management. CMM emphasizes continuous improvement ISO deals with minimum criteria of quality systems There is a clear correlation between the key processes in the CMM and the quality management processes in ISO 9000 has little explicit support for continuous improvement Summer 2003 34

ISO vs CMM (2) l l l Topic 14 The CMM is more detailed and prescriptive and includes a framework for improvement An ISO 9001 -compliant organization would not necessarily satisfy all of the CMM level 2 key process areas (it would satisfy most of the level 2 goals and many level 3 goals. Organisations rated as level 2 in the CMM are likely to be ISO 9000 compliant Summer 2003 35

ISO 9000 and CMM compared CMM ISO 9001 Specific to software development Intended for most industries Used in USA, less widely elsewhere Recognised and accepted in most countries Provides detailed and specific definition of what is required for given levels Specifies concepts, principles and safeguards that should be in place l Assesses on 5 levels Establishes one acceptable level l CMM Level 2 - 3 ISO 9000 Relevant to s/w development process Stabilises the customer - supplier relationship l No time limit on certification Certification valid for three years l No ongoing audit Auditors may return for spot checks during the lifetime of the certificate l l Topic 14 Summer 2003 36

Personal Software Process (PSP) Developed by SEI in 1994 l. A measurement and analyses framework to help you characterize your process l. A defined procedure to help you to improve your performance l. PSP principles l • System quality depends on the quality of its worst components • Component quality depends on individual developers Topic 14 Summer 2003

Overview of CMM and PSP l l Topic 14 CMM sets out the principal practices for managing the processes in large-scale software development PSP sets out the principal practices for defining, measuring and analysing an individual’s own processes Summer 2003 38

PSP l PSP applies a CMM-like assessment for individual work • Measurement & analysis framework to help you characterize your process » Self-assessment and self-monitoring • Prescribes a personal process for developing software » defined steps » Forms » Standards • Assumes individual scale & complexity » Well-defined individual tasks of short duration Topic 14 Summer 2003 39

PSP Overview The PSP is introduced in 7 upward compatible steps (4 levels) l l Write 1 or 2 small programs at each step • Assume that you know the programming language l Gather and analyze data on your work • Many standard forms & spreadsheet templates l Topic 14 Use these analyses to improve your work • Note patterns in your work Summer 2003

PSP Evolution Cyclic Personal Process PSP 3 Cyclic development PSP 2 Personal Quality Management Code reviews Design reviews PSP 1 Personal Planning Process Baseline Personal Process Topic 14 Size estimating Test report PSP 0 Current process Time recording Defect type standard Summer 2003 PSP 2. 1 Design templates PSP 1. 1 Task planning Schedule planning PSP 0. 1 Coding standard Size measurement Process improvement proposal (PIP)

PSP Evaluation l Humphrey has used in SE courses • Improvements in time-to-compile, quality and productivity l Patchy, but promising use in industry • E. g. Nortel (Atlanta) l l Still immature Requires large overhead for data gathering • Not clear that you should use permanently or continually Topic 14 Summer 2003

PSP/TSP/CMM Topic 14 Summer 2003 43