Measurement and Analysis Practical Software and Systems Measurement

Measurement and Analysis Practical Software and Systems Measurement A foundation for objective project management PSM M&A - 1 COCOMO Forum October 24, 2001 24 Oct 01

Measurement and Analysis Outline • Objectives of Measurement and Analysis Initiatives • Practical Software and Systems Measurement • ISO/IEC 15939, Software Measurement Process • CMMI Measurement and Analysis Process Area • ISO 9000 -3, Guidelines for Software M&A - 2 24 Oct 01

Measurement and Analysis Objectives of Measurement Initiatives • Experienced-Based Measurement to Support Management and Engineering Requirements • Consistent Measurement Process Across Measurement Initiatives • Consistent Measurement Terminology M&A - 3 24 Oct 01

Measurement and Analysis Practical Software and Systems Measurement ISO 9000 -3 INFORMATION ISO/IEC DRIVEN MEASUREMENT 15939 PROCESS CMMI M&A - 4 24 Oct 01

Measurement and Analysis Measurement Process Model Measurement Requirements Information Needs Technical & Management Processes User Feedback Information Products Core Measurement Process Establish Commitment Plan Measurement Plan Perform Measurement Experience Base Performance Information Evaluate Measurement Evaluation Results Improvement Actions Measurement Process Scope M&A - 5 24 Oct 01

Measurement and Analysis Practical Software and Systems Measurement (PSM) M&A - 6 24 Oct 01

Measurement and Analysis What is PSM? • A Primary Measurement and Analysis Process Used by Do. D, Government, and Commercial Programs • Technology Based on Actual Experience on Real Programs • A Fully Integrated Approach Including Guidance, Tools, Training, Implementation Workshops, Support Products, and a Qualified Technical Team M&A - 7 24 Oct 01

Measurement and Analysis Key Measurement Concepts • Measurement is a Process - Not a Pre-Defined List of Graphs or Reports • The Measurement Process is Flexible Adapted To Meet Specific Project and Organizational Objectives and Information Needs • The Measurement Requirements are Integrated Into the Project Technical and Management Processes M&A - 8 24 Oct 01

Measurement and Analysis Common Information Needs • • M&A - 9 Schedule and Progress Resources and Cost Product Size and Stability Product Quality Process Performance Technology Effectiveness Customer Satisfaction 24 Oct 01

Measurement and Analysis Problem Report Status Priority 1, 2, and 3 Problem Reports Build 1 1000 900 Start of Build 1 I&T Number of Problem Reports 800 700 600 Discovered Closed 500 400 300 200 100 0 Oct 97 Jan 98 Apr 98 Jul 98 Oct 98 Jan 99 Apr 99 30 Aug 98 M&A - 10 24 Oct 01

Measurement and Analysis Types of Analysis Estimation Performance Analysis M&A - 11 Defined Information Needs Feasibility Analysis 24 Oct 01

Measurement and Analysis Estimation M&A - 12 24 Oct 01

Measurement and Analysis Integrated Analysis Model M&A - 13 24 Oct 01

Measurement and Analysis Integrated Analysis Example M&A - 14 24 Oct 01

Measurement and Analysis Multi-Level Requirements Enterprise Management • • Performance Measurement Normative Performance Baselines Technical and Business Policy Investment Decisions & Analysis Organizational Management • • Process Improvement Program Planning Guidelines Performance Based Acquisition Organizational Norms & Benchmarks Program Management • • Program Estimation and Planning Program Performance Tracking Program Tradeoff Analysis Resource Management M&A - 15 Risk Management Process Issue-Driven Measurement Process 24 Oct 01

Measurement and Analysis PSM Project Strategy Government - Industry Academia Team tical Prac re a Softw ment ure Meas Proven Measurement Technical Guidance and Tools Comprehensive Transition Support M&A - 16 Technical Consensus of Best Practices PSM Project Approach Incremental Product Development and Improvement Empowered Transition Organizations Integrated With Other Key Initiatives 24 Oct 01

Measurement and Analysis New Technology Areas • Measurement and New Technologies - Object-oriented developments Spiral/evolutionary approaches Interoperability Product lines • Initial Recommendations Were Drafted in Early 2001 • Field Trials Are Underway - Additional Trials Will be Conducted in 2002 M&A - 17 24 Oct 01

Measurement and Analysis PSM Participating Organizations Overseas Do. D and Government • US Air Force AFMC • US Air Force SSSG • US Air Force STSC • US Air Force CRSIP • US Air Force STRATCOM • US Army AMC • US Army ATEC • US Army CECOM • US Army ODISC 4 • US Army SMDC • US Army TACOM - ARDEC • USMC MCTSSA • US Navy AEDC • US Navy ASN RDA • US Navy FMSO • US Navy NAVAIR • US Navy NAVSEA • US Navy NAWC • US Navy NRL • US Navy NSWC M&A - 18 • US Navy NUWC • US Navy OPTEVFOR • ODUSD (S&T) • OSD C 31 • OSD PA&E • Aerospace Corp. • DCMA • DFAS • DISA • DLA • DSMC • FAA • IDA • MITRE Corp. • NDU IRM College • NASA • National Park Service • Sandia National Lab • SEI • Soc. Sec. Admin. • Defence Materiel Org. (Australia) • Live. Ware (Argentina) • NRC (Canada) • Software and Systems Engineering (Israel) • Software Improvements (Australia) • Software Quality Institute (Australia) 24 Oct 01

Measurement and Analysis PSM Participating Organizations Industry • ACS GSG • American Sys. Corp. • Averstar • Bloodworth Int. Tech. • Carnegie Mellon Univ. • CSC • CTA • Distributive Software • Federal Reserve Bank • General Dynamics • General Scientific Corp. • Graeme & Garland • GTE • Hawaiian Electric • IEEE • IFPUG • IITRI • INCOSE M&A - 19 • Independent Engineering • ITABHI • James Gregory Assoc. • Kodak Health Imaging • Litton/TASC • Lockheed Martin • L 3 Communications • Northrop Grumman • OAO Corporation • PRC - Litton • PRC/TASC • QSM • Quality Plus Tech. • Raytheon - Hughes • Robbins, GIOIA • Rockwell Collins • SAIC • Sallie Mae • Sentel • Software Metrics • SPC • Sverdrup Tech. • Tecolote Research • Tera. Quest Metrics • Tivoli • TRW • United Defense • Univ. Southern CA • User Trust Network • US West • VPI - State University • West Virginia High Tech. Cosortium • West Virginia University 24 Oct 01

Measurement and Analysis M&A - 20 24 Oct 01

Measurement and Analysis ISO/IEC 15939, Software Measurement Process M&A - 21 24 Oct 01

Measurement and Analysis ISO/IEC 15939 • PSM Was Base Document for This Standard • Specifies Required Activities and Tasks of a Measurement Process • FCD Ballot Passed in May 2001 With Only 1 Dissenting Vote - All Comments Resolved at ISO Plenary • Terminology Was Adjusted for Consistency With ISO/IEC 9126, 15498, and 14143 Series • DIS Ballot Will be Released in September 2001 - Ballot Will Close in November 2001 International Standard in Late 2001 M&A - 22 24 Oct 01

Measurement and Analysis Basic Measurement “Architecture” ISO/IEC 15939 Measurement Information Model Measurement Process Model Information Driven Measurement Requirements Consistent, Tailorable Implementation Process M&A - 23 24 Oct 01

Measurement and Analysis Measurement Information Model Information Needs Measurable Concept Information Product Measurement Construct Entities Attribute M&A - 24 24 Oct 01

Measurement and Analysis Information Needs Indicator Analysis Model Derived Measure Meas. Function Quantity Defined as a Function of Two or More Measures Algorithm Combining Two or More Measures Base Measure Meas. Method M&A - 25 Algorithm Combining Measures and Decision Criteria Derived Measure Base Measure Attribute Measure that Provides an Estimate or Evaluation With Respect to Information Needs Attribute Measure of a Single Attribute Defined By a Specified Measurement Method Operations Mapping an Attribute to a Scale Property Relevant to Defined Information Needs Entities 24 Oct 01

Measurement and Analysis Information Product Information Needs Information Product Decision Criteria Indicator Analysis Model Base Measure M&A - 26 Measure that Provides an Estimate or Evaluation With Respect to Information Needs Algorithm Combining Measures and Decision Criteria Derived Measure 24 Oct 01

Measurement and Analysis CMMI - Measurement and Analysis (M&A) M&A - 27 24 Oct 01

Measurement and Analysis CMMI M&A • ISO/IEC 15939 Was Base Document for the Measurement and Analysis Process Area • CMMI M&A Describes How to Evaluate a Measurement Process • M&A Is a Level 2 Process Area • Pilot Assessments Are Being Conducted in 2001 M&A - 28 24 Oct 01

Measurement and Analysis ISO 9000 -3 Quality management and quality assurance standards - Part 3: Guidelines for the application of ISO 9001 to the development, supply, installation, and maintenance of computer software M&A - 29 24 Oct 01

Measurement and Analysis ISO 9000 -3 • 9000 -3 Is Being Updated by SC 7 as Part of the ISO 9000: 2000 Update • Part 8, “Measurement, Analysis and Improvement”, Was Updated for Consistency With 15939 • Standard Is at Working Draft Stage - Expected Publication Date Is 2002/2003 M&A - 30 24 Oct 01

Measurement and Analysis For More Information PSM Support Center Cheryl Jones - PSM Project Manager TACOM - ARDEC AMSTA-AR-QAT Building 62 Picatinny Arsenal, NJ 07806 -5000 (973) 724 -2644 (Voice) (973) 724 -2382 (FAX) cljones@pica. army. mil PSM Home Page: http: //www. psmsc. com M&A - 31 24 Oct 01