The CMM Integration Project Dr Jack R Ferguson

The CMM Integration Project Dr. Jack R. Ferguson Dr. Rick Hefner CMMI Project Manager Assessment Team Co-Lead 1

Objectives • Present the background and current status of the CMM Integration Project • Discuss structure and sample content of the new maturity models • Discuss timeline for public release of the models and pilot assessments • Discuss transition from the current maturity models and assessment methods 2

Agenda • Introduction Dr. Jack Ferguson (SEI) • Background • Design Approach • Comparison to SW-CMM v 1. 1 (TRW) • Comparison to EIA IS 731 (SECM) • Assessment Methodology • Comment Process • Transition Process • Discussion Dr. Rick Hefner Dr. Jack Ferguson (SEI) 3

Background Dr. Jack Ferguson Objectives • Review project objectives • Review key requirements and source material • Discuss CMMI project team • Compare and contrast current maturity models – CMM for Software, SECM, IPD-CMM – Staged, continuous 4

What are Capability Maturity Models? • Organized collections of best practices • Based on work by Crosby, Deming, Juran, Humphrey. . . • Systematic ordered approach to process improvement. • Means of measuring organizational maturity. • Have proven to bring significant return on investment in productivity and quality. 5

How are CMMs used? • Process Improvement • Process Definition • Competency Assessment • Risk Management • Communication 6

The Current Situation - every silver lining has a dark cloud • Explosion of CMMs and CMM-like models • Multiple assessments • Multiple training • Multiple models within • Multiple expenses an organization 7

Courtesy Sarah Sheard, SPC 8

Why is This a Problem? • Similar process improvement concepts, but. . . – Different model representations (e. g. staged, continuous, questionnaire, hybrid) – Different terminology – Different content – Different conclusions – Different appraisal methods 9

The Common Basis for Model. Based Process Improvement in any discipline is a function of performing: – Implementing practices that reflect the fundamentals of a particular topic (e. g. configuration management) – Institutionalizing practices that lead to sustainment and improvement of an implementation 10

All CMMI source models contain: • Implementing practices grouped by affinity • Institutionalizing practices that vary from model to model, however all models specify levels that describe increasing capability to perform 11

Improvement Levels Continuously improving process Quantitatively (4) Managed Predictable process Standard, consistent process Defined Managed Disciplined process Initial Not performed (0) (1) Optimizing (2) (3) (5) (measured) (standard) (planned and tracked) (performed) 12

The CMMI Project • Do. D sponsored • Collaborative endeavor – Industry – Government – Academia • Over 100 people involved 13

CMM Integration Project Steering Group Co-Chairs P. Babel / B. Rassa Stakeholder/ Reviewers Product Development Team Project Manager J. Ferguson Coordinating IPT Team leads Architecture IPT PA Author Groups Usability Team IPPD IPT Training Meth. IPT Lotus Notes Team Chief Architect R. Bate Editors (CCB) Assessment Meth. IPT Reqmts. IPT Pilot Planning 14

The CMMI Development Team • U. S. Air Force • U. S. Navy • Federal Aviation Administration • National Security Agency • Software Engineering Institute (SEI) • ADP, Inc. • Boeing • Computer Sciences Corp. • • • Ericsson Canada General Dynamics Honeywell Litton Lockheed Martin Northrop Grumman • • • Pacific Bell Raytheon Rockwell Collins Thomson CSF TRW 15

CMMI Design Goals • Integrate the models, eliminate inconsistencies, reduce duplication • Reduce the cost of implementing modelbased process improvement • Increase clarity and understanding – Common terminology – Consistent style – Uniform construction rules – Common components • Assure consistency with ISO 15504 • Be sensitive to impact on legacy efforts 16

Benefits • Efficient, effective assessment and improvement across multiple process disciplines in an organization • Reduced training and assessment costs • A common, integrated vision of improvement for all elements of an organization • A means of representing new discipline-specific information in a standard, proven process improvement context 17

The Challenge • Extract the common or best features from the source models • Provide users the ability to produce single- or multiplediscipline models, both continuous and staged, tailored to their organizational needs. • Provide users the ability to assess and train based on these models. 18

CMMI Source Models • Capability Maturity Model for Software V 2, draft C (SW-CMM V 2 C) • EIA Interim Standard 731, System Engineering Capability Model (SECM) • Integrated Product Development Capability Maturity Model, draft V 0. 98 (IPD-CMM) 19

Staged Representations • Key Process Areas are grouped in the stages (levels) from 2 to 5 • A Key Process Area contains specific practices (activities) to achieve the purpose of the process area. • For a Key Process Area at a given stage, institutionalization practices are integral to the process area. 20

Staged Model Level Key Process Areas Org Improvement Deployment Continuous 5 Org Process and Tech Innovation process Optimizing improvement Defect Prevention 4 Organization Process Performance Quantitatively Quantitative Statistical Process Management Managed management Org Software Asset Commonality Peer Reviews 3 Process Project Interface Coordination Defined Standardization Software Product Engineering Organization Training Program Organization Process Definition Organization Process Focus Software Configuration Management 2 Basic Repeatable Software Quality Assurance Project Software Acquisition Management Software Project Control Software Project Planning Requirements Management 1 Initial Focus Competent people and heroics 21

Continuous Representations • A process area contains specific practices to achieve the purpose of the process area. • Generic practices are grouped in Capability Levels • Generic practices are added to the specific practices of each process area to attain a capability level for the process area. • The order in which Process Areas are addressed can follow a recommended staging. 22

Continuous Model 23

Source Model Terminology 24

Assessment Methods • CBA IPI Method – – Rating of goals Single digit rating Full goal satisfaction More strict data validation requirement • SECM Assessment Method – – Rating of practices Granularity options Partial credit options Less strict data validation requirement Both methods use the same basic set of activities 25

The CMMI Challenge • Integrate three source models that have many differences • Provide consistency with ISO 15504 • Maintain support from user communities • Develop framework to allow growth to other disciplines 26

Design Approach Objectives • Review design goals • Discuss framework of CMMI models • Describe CMMI terminology and components • Outline CMMI products • Discuss CMMI Schedule and current issues 27

The CMMI Solution A Product Line Approach SW IPPD SE . . . Industry SEI CMMI Product Suite Government Assess Training • Team of Teams • Modeling and Discipline Experts • Collaborative Process CMMI- CMMISW SE SE/SW CMMISE/SW/ IPPD . . . 28

The CMMI Product Line The CMMI product line is a product suite sharing a common, managed set of features that satisfy specific needs of a selected domain. pertain to Domain is satisfied by share an Architecture Products guides development of are built from Components 29

CMMI Product Suite Framework Input • Common content • Discipline content • Criteria for content Transform • Rules for generating products -Model -Assessment -Training Output • Integrated model(s) • Assessment method(s) • Training materials Repository 30

Framework • Components • Construction rules • Conceptual architecture 31

The CMMI Framework General • Glossary • Development Standards and Guidelines • Document Templates • Assessment Methodology • Framework Training Materials Process Management Core (PMC) Integration Core (IC) IPPD Environment PMC Generic Practices/Templates PMC Process Areas PMC Assessment Material PMC Model Training Material PMC Assessment Training Material IC Generic Practices/Templates IC Process Areas IC Assessment Material IC Model Training Material IC Assessment Training Material Shared (Discipline X+Y) Discipline Y Discipline X (DX) + DX Amplifications DX Process Areas DX Assessment Material DX Model Training Material DX Assessment Training Material Output Models Assessment Methods Model Training Materials Assessment Training Materials 32

CMMI Terminology CMMI Models contain institutionalization (Generic) and implementation (Specific) parts: • Front matter • Process Areas that contain: Required – Generic and Specific Goals Expected – Generic and Specific Practices (in Common Features in staged representation) Informative – Subpractices – Notes – Discipline-specific amplifications • Glossary and tailoring guidelines 33

CMMI V 0. 2 Process Areas - 1 Maturity Level 2 Process Management Core • Project Planning Engineering Shared (SE & SW) Requirements Management • Project Monitoring and Control • Configuration Management • Process & Product Quality Assurance • Supplier Agreement Management • Data Management • Measurement & Analysis 34

CMMI V 0. 2 Process Areas - 2 Maturity Level 3 Process Management Core Engineering Shared (SE & SW) • Organizational Process Focus • Customer & Product Requirements • Organizational Process Definition • Organizational Training • Technical Solution • Product Integration • Integrated Project Management • Product Verification • Risk Management • Validation • Decision Analysis & Resolution 35

CMMI V 0. 2 Process Areas - 3 Maturity Levels 4 & 5 Process Management Core • Quantitative Management of Quality and Process • Organizational Process Performance • Causal Analysis and Resolution • Organizational Process Technology Innovation • Process Innovation Deployment 36

CMMI Products • CMMI Models • Assessment Material • Training Material • Model Developer Material 37

CMMI Models • Staged and Continuous (with equivalent staging) versions of: – Software Engineering – Systems Engineering + Software with IPPD • Tailoring Guidance 38

Assessment Material • Assessment requirements • Assessment methodology • Assessment data collection methods and tools (e. g. , questionnaires, interviews) • Assessment Team qualifications 39

Training Material • Model Training • Assessment Training – Team Training – Lead Assessor Training 40

Model Developer Material • Glossary • Framework and model content criteria • Framework Training 41

CMMI Schedule • August 31, 1999 Release CMMI-SE/SW V 0. 2 public review. for • Nov 30, 1999 public review for Release CMMI-SE/SW/IPPD • Nov 1999 -May 2000 • Jun-Aug 2000 Pilot assessments Publish models V 1. 0 42

Issues Concerning Initial CMMI Drafts • Size of model • Complexity of model • “Normative” model • Goals and Themes • Order of process areas • Equivalence between staged and continuous representations – “Advanced” practices – Process area boundaries – Generic practices • ISO Consistency CMMI team received 4000+ change requests from reviewers 43

CMMI-SE/SW compared to SW-CMM v 1. 1 Dr. Rick Hefner Objectives • Philosophy • Model Component Comparison • Process Area Comparison • Common Features Comparison 44

Philosophy - 1 • SEI had completed updates to the SW-CMM when the CMMI project was started – SW-CMM v 2 Draft C was used as the source model for CMMI – Adapted for compatibility with SE • Most of the community is currently using SW-CMM v 1. 1 – Detailed traceability matrices are being developed 45

Philosophy - 2 • CMMI- SE/SW staged representation is similar to SW-CMM v 1. 1 – Maturity Levels composed of Process Areas – Goals are required; implemented & institutionalized – Key practices are expected; alternative practices are acceptable if effective at meeting the goals – All else is informative • CMMI- SE/SW continuous representation reflects the same info in a SPICE-like structure 46

Model Component Comparison CMMI Models SW CMM Process Area Key Process Area Generic Goal Planning Goal sometimes used v 1. 1, Institutionalization Goal in 2. 0 Specific Goal KPA Goal Generic Practice Key practices from institutionalization common features Specific Practice Key Practice from Activities Performed Common Features Subpractice Maturity Level Capability Level Notes None Explanatory Material Work Products Examples “For Software Engineering” Examples and explanatory materiel 47

SW-CMM v 1. 1 CMMI LEVEL 4 MANAGED LEVEL 3 DEFINED Defect prevention Technology change mgmt Process change mgmt Causal Analysis and Resolution Org. Process Technology Innovation Process Innovation Deployment Quantitative process mgmt Software quality mgmt Org. Process Performance Quantitative Mgmt of Quality & Process Organization process focus Organization process definition Training program Integrated software mgmt LEVEL 5 OPTIMIZING Organization process focus Organization process definition Organizational training Integrated project management Risk Management Customer and Product Reqmts Technical Solution Product Integration Product Verification Validation Decision Analysis and Resolution Software product engr Intergroup coordination Peer reviews LEVEL 2 REPEATABLE Requirements management Software project planning Software project tracking & oversight Software subcontract mgmt Software quality assurance Software configuration mgmt Requirements management Project planning Project Monitoring and Control Supplier Agreement Management Product & Process Quality Assurance Configuration Management Data Management 48 Measurement and Analysis

Software Product Engineering SW-CMM v 1. 1 Activities 1 Appropriate software engineering methods and tools are integrated into the project's defined software process. 2 The software requirements are developed, maintained, documented, and verified by systematically analyzing the allocated requirements according to the project's defined software process. 3 The software design is developed, maintained, documented, and verified, according to the project's defined software process, to accommodate the software requirements and to form the framework for coding. 4 The software code is developed, maintained, documented, and verified, according to the project's defined software process, to implement the software requirements and software design. Customer and Product Req PA Technical Solution PA Product Verification PA 49

Software Product Engineering SW-CMM v 1. 1 Activities (continued) 6 Integration testing of the software is planned and performed according to the project's defined software process. Product Integration PA 7 System and acceptance testing of the software planned and performed to demonstrate that the software satisfies its requirements. Product Verification PA 8 The documentation that will be used to operate and maintain the software is developed and maintained according to the project's defined software process. 9 Data on defects identified in peer reviews and testing are collected analyzed according to the project's defined software process. 10 Consistency is maintained across software work products, including the software plans, process descriptions, allocated requirements, software design, code, test plans, and CMMI 50

Common Feature Comparison Handled by the Measurement and Analysis PA 51

Conclusions • Organizations using SW-CMM v 1. 1 should be able to smoothly transition to CMMI – Measurement and Analysis & Data Mgmt at L 2 – Risk Management & Decision Analysis and Resolution at L 3 – Expansion of Software Product Engineering – Configuration Management for all Process Areas 52

Comparing CMMI-SE/SW to EIA IS 731 -SECM Objectives • Philosophy • Process Area Comparison • Planned IPPD Extensions 53

Philosophy - 1 • EIA 731 was created as a merger of the SECMM and INCOSE SECM models – Used as a source model for CMMI • CMMI-SE/SW merges software ideas – Staged representation of SE available – Continuous representation with “equivalent staging” 54

55

EIA 731 Focus Areas - 1 • Technical Focus Areas – Define Stakeholder and System Level Requirements – Define Technical Problem – Define Solution – Assess and Select – Integrate System – Verify System – Validate System Causal Analysis and Resolution (PM) Org Process Technology Innovation (PM) Process Innovation Deployment (PM) Quantitative Mgmt of Quality and Process (PM) Organizational Process Performance (PM) Organizational Process Focus (PM) Organizational Process Definition (PM) Organizational Training (PM) Integrated Project Management (PM) Risk Management (PM) Decision Analysis and Resolution (PM) Customer and Product Requirements (Eng) Technical Solution (Eng) Product Integration (Eng) Product Verification (Eng) Validation (Eng) Project Planning (PM) Project Monitoring and Control (PM) Configuration Management (PM) Product and Process Quality Assurance (PM) Supplier Agreement Management (PM) Data Management (PM) Measurement and Analysis (PM) Requirements Management (Eng) 56

EIA 731 Focus Areas - 2 • Management Focus Areas – – – – Plan and Organize Monitor and Control Integrate Disciplines Coordinate with Suppliers Manage Risk Manage Data Manage Configurations Ensure Quality Causal Analysis and Resolution (PM) Org Process Technology Innovation (PM) Process Innovation Deployment (PM) Quantitative Mgmt of Quality and Process (PM) Organizational Process Performance (PM) Organizational Process Focus (PM) Organizational Process Definition (PM) Organizational Training (PM) Integrated Project Management (PM) Risk Management (PM) Decision Analysis and Resolution (PM) Customer and Product Requirements (Eng) Technical Solution (Eng) Product Integration (Eng) Product Verification (Eng) Project Planning (PM) Project Monitoring and Control (PM) Validation (Eng) Configuration Management (PM) Product and Process Quality Assurance (PM) Supplier Agreement Management (PM) Data Management (PM) Measurement and Analysis (PM) Requirements Management (Eng) 57

EIA 731 Focus Areas - 3 • Environment Focus Areas – Define and Improve the Systems Engineering Process – Manage Competency – Manage Technology – Manage Systems Engineering Support Environment Causal Analysis and Resolution (PM) Org Process Technology Innovation (PM) Process Innovation Deployment (PM) Quantitative Mgmt of Quality and Process (PM) Organizational Process Capability (PM) Organizational Process Focus (PM) Organizational Process Definition (PM) Organizational Training (PM) Integrated Project Management (PM) Risk Management (PM) Decision Analysis and Resolution (PM) Customer and Product Requirements (Eng) Technical Solution (Eng) Product Integration (Eng) Product Verification (Eng) Validation (Eng) Project Planning (PM) Project Monitoring and Control (PM) Configuration Management (PM) Product and Process Quality Assurance (PM) Supplier Agreement Management (PM) Data Management (PM) Measurement and Analysis (PM) Requirements Management (Eng) 58

What’s New? • Not a EIA 731 Focus Area (but in the content) – Causal Analysis and Resolution – Process Innovation Deployment – Quantitative Process and Quality Mgmt – Organizational Process Performance Causal Analysis and Resolution (PM) Org Process Technology Innovation (PM) Process Innovation Deployment (PM) Quantitative Mgmt of Quality and Process (PM) Organizational Process Performance (PM) Organizational Process Focus (PM) Organizational Process Definition (PM) Organizational Training (PM) Integrated Project Management (PM) Risk Management (PM) Decision Analysis and Resolution (PM) Customer and Product Requirements (Eng) Technical Solution (Eng) Product Integration (Eng) Product Verification (Eng) Validation (Eng) Project Planning (PM) Project Monitoring and Control (PM) Configuration Management (PM) Product and Process Quality Assurance (PM) Supplier Agreement Management (PM) Data Management (PM) Measurement and Analysis (PM) Requirements Management (Eng) 59

Conclusions • EIA 731 users should be able to smoothly transition to the CMMI-SE/SW model – Continuous representation (+ “equivalent” staged representation) – Some lower level differences • Integrated Product and Process Development (IPPD) will be added – Based on IPPD-CMM practices 60

Assessment Methodology Dr. Rick Hefner (TRW) Assessment Methodology Team Co-lead Objectives • Assessment approach • Assessment Requirements for CMMI (ARC) • SCAMPI assessment method • Lead Assessor program, transition plan 61

Assessment Methods • CBA IPI Method – – Rating of goals Single digit rating Full goal satisfaction More strict data validation requirement • SECM Method – – Rating of practices Granularity options Partial credit options Less strict data validation requirement Essentially the same activities in the two methods 62

Assessment Requirements for CMMI (ARC) • Similar to the current CMM Appraisal Framework (CAF) V 1. 0 – Specifies the minimum requirements for full, comprehensive assessment methods, e. g. , SCAMPI • Other assessment methods may be defined for situations not requiring a comprehensive assessment – initial assessment, quick-look, process improvement monitoring, etc. 63

Standard CMMI Assessment Method for Process Improvement (SCAMPI) • Similar to CBA IPI method • Led by authorized Lead Assessor • Tailorable to organization and model scope • Artifacts: – SCAMPI Method description document – Maturity questionnaire, work aids, templates • Current activities – Merger of SECM appraisal method features 64

CMMI Lead Assessor Program • Similar to existing SEI Lead Assessor and Lead Evaluator programs • Grandfather current Lead Assessors • Under consideration – Delineate by discipline, e. g. , SW Lead Assessors, SE Lead Assessors? – Details of transition process for current Lead Assessors and other assessment leaders – Required training in CMMI models 65

Comment Process Dr. Jack Ferguson • Release CMMI-SE/SW v 0. 2 August 31 – Available at http: //www. sei. cmu. edu – Public comments due November 30 • Release CMMI-SE/SW/IPPD November 30 – Comments due February 28 • Hold Focus Group discussions – SEI Transition – Assessors for both communities – SPINs 66

Pilot Assessments • Nine initial assessments (desired) – Supported by 3 Product Development Team (PDT) members, – Covering all CMMI models, staged and continuous representations – Nine organizations (4 Do. D & 5 industry) volunteered to participate • • • 1 - CMMI SE/SW with IPPD, staged or continuous representation 4 - CMMI SE/SW, staged representation 2 - CMMI SE/SW, continuous representation 1 - CMMI SE, continuous representation 1 - CMMI SW, staged representation • Product Development Team (PDT) member roles – CMMI Product Suite Training – Coaching and structured observation • Structured feedback from assessment participants – Assessors and Sponsors, and – Participating organization members 67

CMMI Transition Plan • Development Phase – Development of CMMI products – Verification and Validation of CMMI products • Transition Phase – Approval of a CMMI Product for public release – Evidence of sufficient use – Transition planning to help organizations use CMMI products • Sustainment Phase – Upkeep & continuous improvement of the product suite – Additional evidence of adoption and use Sustainment Phase Transition Phase Development Phase 68

Transitioning to Use of CMMI • Understand how models are used: – Steps to enterprise-wide process improvement • Apply Lessons Learned in transitioning from single-discipline models – Federal Aviation Administration’s experiences with i. CMM – US Air Force experiences with transitioning between models – Others. . . 69

Steps to Enterprise-wide Organizational Maturity 1. Vision, Goals, Buy-in Organization Processes 2. Process Capture 4. Process Transformation Project Management Process Level Evolution Best Practices 3. Gap Analysis From Process, all good things flow! Metrics Capture Activity Based Costing Simulation 70

CMMI Benefits • CMMI product users can expect to: – Efficiently and effectively improve and assess multiple disciplines across their organization – Reduce costs (including training) associated with improving and assessing processes – Deploy a common, integrated vision of Improved Processes process improvement that can be used as a basis for enterprise-wide process improvement efforts. 71

The promise. . . • CMMI team is working to assure the CMMI Product Suite addresses needs of software and systems engineering communities of practice • Use of an integrated model to guide enterprise process improvement promises to be one of the more sustainable & profitable initiatives that any organization might pursue 72

Discussion 73