CECIM Cost Engineering Capability Improvement Model Association of

CECIM Cost Engineering Capability Improvement Model Association of Cost Engineers 17 February 2004 Hugh Pickerin VORTICITY LTD Hampden House Warpsgrove Lane Chalgrove OXON OX 44 7 RW hugh. pickerin@vorticity-systems. com +44 (0)1865 -893210, +44 (0)7802 -834794 1

CECIM Objectives • To encourage the application of bestpractice methodologies & techniques – Business Improvement Model • To define the scope of Cost Engineering • To define the competences and functions that comprise cost engineering © 2002 Hugh Pickerin & Dave Lewis 2

Why CECIM? ? Improve the profile of Cost Engineering by: • Describing the CE Body of Knowledge • Providing an identity for the CE domain & for cost engineering practitioners themselves © 2002 Hugh Pickerin & Dave Lewis 3

Why “CECIM”? CECIM = (Cost Engineering) + (Capability Improvement Model) = (better value) +

The Purpose of CECIM • CECIM offers a means by which organisations can assess their current CE capability and so determine what actions need to be taken to improve their capabilities • For organisations without a formal approach to CE, it provides a means of becoming familiar with the activities that constitute best practice © 2002 Hugh Pickerin & Dave Lewis 5

The Purpose of CECIM /cont • CECIM provides a means to evaluate an organisations capability. It provides a 6 step ladder, as a means of defining and quantifying process improvement • It can be used in whole or in part, depending on organisational needs • Higher levels may not always be justified - too expensive for some organisations? © 2002 Hugh Pickerin & Dave Lewis 6

What is Cost Engineering? We propose the following definition: the basic ‘Hard’ skills that underpin successful programme management, including: – Estimating ( - cost, resource, time) – Cost Control – Planning (i. e. scheduling ~ time estimation) – Schedule Control – Cost & Schedule Risk Management © 2002 Hugh Pickerin & Dave Lewis 7

CECIM Methodology Objectives A process framework for planning & estimating: • • Better forecasting accuracy Improved consistency Improved competitiveness Better informed ‘go – no go’ decisions for projects © 2002 Hugh Pickerin & Dave Lewis 8

CECIM Model Objectives • Provides a benchmark for ‘hard’ PM skills • Aids achievement

Rationale –PM methodologies don’t fully address ‘hard’ skills –Wide variations in capability observed –Outturn variance is sensitive to estimating capability –Over-optimistic estimates mask programme risks –‘Fat’ estimates will deliver, but not competitively So, We need a formal process framework for defining and improving Cost Engineering capability © 2002 Hugh Pickerin & Dave Lewis 10

CECIM Development • launch followed authors’ EACE initiative (EACE = European Aerospace Cost Engineering Working Group) • model inputs from: • Airbus UK • Cost Engineering Solutions • Anglian Enterprises • ESA/ESTEC • ASI • Mo. D DPA/PFG • Astrium • Westland Helicopters • BAe Systems • self-funded by participants and/or their employers • CECIM White Paper published May 2001 © 2002 Hugh Pickerin & Dave Lewis 11

CECIM Architecture reproduced from SEI SE-CMM with permission 12

The Domain Portion “What Should The Organisation Be Doing? ” Defines the processes of the CE discipline related to: – The discipline itself – Integration within the project – Organisation-wide processes Decomposes the discipline into “Process Areas” (PAs) – Decomposes the PAs into “Base Practices” (BPs) reproduced from SEI SE-CMM with permission 13

The Capability Portion – “How Well Is It Performing? ” • Defines 6 increasing levels of process maturity • Defines generic practices for each Capability Level adapted from SEI SE-CMM with permission 14

Domain + Capability = Maturity reproduced from SEI SE-CMM with permission 15

CECIM Model Contents Unique Domain Portion: • • 19 Process Areas 101 Cost Engineering BPs defined Typical Work Products identified for each BP Links between BPs established Domain portion architecture follows SEI format Generic Capability Portion: • features 5 Capability Levels (plus level 0) • comprises 28 common features & generic practices Capability portion adopts standard PI definitions (e. g. SEI) © 2002 Hugh Pickerin & Dave Lewis 16

CECIM Process Areas • • • Cost Estimating Cost Modelling Cost Control & Analysis VA/VE & Cost Reduction Planning Risk Management Competences Management Define Process Improve Process Integrate Disciplines • • • Ensure Quality DTC & CAIV Supply Chain Knowledge Management Capital Asset & Resource Management Business Analysis Business Case Development Audit Cost Allocation © 2002 Hugh Pickerin & Dave Lewis 17

PA 01: Cost Estimating • Understand customer needs • Classes of estimates • Develop metrics • Create WBS, OBS, PBS, CBS • Generate target costs • Determine preliminary schedule • Generate make/buy plan • Obtain material costs • Generate detailed estimates for in house work • Agree foreign exchange rates • Compile works cost • Calculate contingency • Utilisation of learning curve • Issue definitive estimate © 2002 Hugh Pickerin & Dave Lewis 18

PA 02: Cost Modelling • • Preliminary model design Identify system information requirements Data collection Data analysis Develop models Calibrate and validate models Model maintenance © 2002 Hugh Pickerin & Dave Lewis 19

PA 03: Cost Control & Analysis • • Establish requirements Confirm WBS Revise estimate Establish baseline (BCWS) Collect costs Performance measurement (including EVA) Estimate to completion Assimilation of Lessons Learned © 2002 Hugh Pickerin & Dave Lewis 20

EACE Cost Engineering 2001 Maturity Survey © 2002 Anglian Enterprises Ltd 21

Survey Background • • survey based on CECIM model PAs and BPs questionnaire designed launched at EACE Frascati May 2001 session requested return date 31 st July 2001 MS Access database built SQL used for analysis tasks correlations, means, sensitivities etc analysed results charted in MS Excel © 2002 Anglian Enterprises Ltd 22

Survey Process • participation free-of-charge to respondents – respondents encouraged to agree to their identification in the published results, subject to non-disclosure of specific data • identified respondents received: – summary data and full analysis report (shows organisational ranking versus other respondents) • anonymous respondents obtained: – summary report (shows summary median, mean, mode and max/min values only) hence, all respondents obtained a useful analysis tool – aid to internal process-improvement activity – basis for presentations to senior management © 2002 Anglian Enterprises Ltd 23

Participants • 18 organisations participated in survey, including a wide range of European space and aerospace organisations: Alenia-Marconi Systems Datamat Sp. A Alenia Difesa - Off. Galileo ESA/ESTEC Astrium - SI Mo. D DPA/PFG BAe Systems © 2002 Anglian Enterprises Ltd 24

Survey Structure # 1 • survey designed in 2 sections: – Section 1 captured details of respondents’ organisations: Type of Organisation Constitution Industry Sector Principal Activities Organisation Size Cost Engineering Organisation Details Respondents Function © 2002 Anglian Enterprises Ltd 25

Survey Structure # 2 • Section 2 invited respondents to rate their organisational capability, by Base Practice, for each Process Area, according to the CECIM Model: Capability Ratings N/A [null] 0 Not Performed 1 Performed Informally 2 Process Planned & Tracked 3 Process Well-Defined 4 Quantitatively Controlled 5 Continuously Improving © 2002 Anglian Enterprises Ltd 26

Analysis Methodology – Base Practices • Maximum, Minimum and Modal values recorded for each Base Practice of all 19 Process Areas – total 101 Base Practices analysed • corresponding arithmetic mean and median values calculated • “N/A” responses treated as nulls and discounted for analysis purposes © 2002 Anglian Enterprises Ltd 27

Benchmark Results – PAs 01 - 10 © 2002 Anglian Enterprises Ltd 28

Benchmark Results – PAs 11 - 19 © 2002 Anglian Enterprises Ltd 29

Results Summary #1 (survey 1. 03) Organisation Structure: – multi-site, multi-unit organisations reported better capability than monolithic enterprises (survey 1. 04) Organisation Entity Type: – Private and Public-Quoted companies reported better capability than Multi-Nationals and Non-Profits (survey 1. 05) Industrial Sector: – overall, no significant differences were observed (survey 1. 06) Principal Activities: – Manufacturers/Developers reported best capability (survey 1. 07) Number of Persons Employed: – generally, larger organisations reported better capability © 2002 Anglian Enterprises Ltd 30

Results Summary #2 (survey 1. 08) Cost Engineering Headcount: – smallest and largest organisations reported better capability (survey 1. 09 a) Dedicated CE Organisation: – dedicated set-ups reported a more consistent and predominantly better capability than their counterparts (survey 1. 09 b) Cost Engineering Responsibility: – in organisations without a dedicated CE set-up, Project/Programme Office predominated (survey 1. 10) Respondents’ Role: – Practitioners generally reported the lowest capability; CE Managers had a more favourable opinion; Other respondents reported the highest capability © 2002 Anglian Enterprises Ltd 31

Conclusions are the results significant? – quantitively, probably not, qualitatively, yes was the survey worthwhile? Yes! – deployed a framework enabling participants & others to embark on process improvement activity – survey respondents and other users can see process improvement opportunities – provides an additional dimension to CECIM – defined a European Cost Engineering benchmark – illustrates potential of CECIM as the basis for future benchmarking activity © 2002 Anglian Enterprises Ltd 32

Recommendations where’s the “low-hanging fruit”? • the following Process Areas are recommended targets for implementation of low-cost improvements: PA 09 Improve the CE Process PA 11 Ensure CE Process Quality PA 14 Knowledge Management PA 15 Capital Asset & Resource Management action in these areas should yield greatest improvement in overall European CE capability plus greatest leverage of CE resources, thus providing maximum benefit to the implementing organisation © 2002 Anglian Enterprises Ltd 33

Lessons Learned • inadequate publicity may have restricted participation lesson: publicise more extensively • the cost - value dichotomy perception that free participation implies no derived value lesson: emphasise BENEFITS! • poor deployment E-deployment was compromised by software version compatibility issues lesson: ensure electronically-transmitted formats are robust (PDF? ) • subjectivity in reporting lesson: external assessment needed for truly-objective results • survey design ok general quality of responses good – few incomplete or otherwise spoiled returns lesson: ensure clear instructions & coherent, user-friendly forms © 2002 Anglian Enterprises Ltd 34

CECIM: Into the 21 st Century © 2002 Hugh Pickerin & Dave Lewis 35

CECIM Current Status • White Paper now at Rev 04 ongoing updates • no external interest expressed in taking ownership • model now “adopted” by Anglian Enterprises and Cost Engineering Solutions responsible for • • maintenance future development deployment (etc) © 2002 Hugh Pickerin & Dave Lewis 36

CECIM: The Future? • repeat survey to calibrate improvements? • survey of US capability? • extend survey range? – within Europe? – worldwide? • extend survey scope? – include sectors outside aerospace & defence? © 2002 Hugh Pickerin & Dave Lewis 37

Towards Best Practice • “owners” forum for Maturity Models? – promote, harmonise approaches & techniques • a moderated CE benchmarking network? – identify best-in-class: share and learn – access to common toolkit: reduced costs – secure environment: safeguards sensitive data • implementation, process improvement & assessment support – Offerings from Vorticity/AEL and Cost Engineering Solutions © 2002 Hugh Pickerin & Dave Lewis 38

An Emerging Standard • Network Rail – CECIM reference model now in use for internal cost engineering capability development. • DPA/PFG – Vendor capability viewed as a significant risk. – 21 -Dec-03: Vorticity/AEL contracted to develop training course for CECIM assessors. – Objective: 40 trained assessors in PFG Field Force. – Methodology extensions for ISO 15504 compliance. • Accreditation – Initial planning underway for a formal accreditation scheme. © 2004 Vorticity Ltd 39

Contacts & References • EACE (UK members of Steering Committee): David Greves (chair), ESA/ESTEC David. Greves@esa. int Carl Dalton, Galorath Affiliates (cdalton@galorath. com) Dave Lewis, Cost Engineering Solutions (admin@costeng-solutions. com) John Henson, Westland (HENSONJ@whl. co. uk) Peter Fray, DPA/PFG (pfgcfair 4@dpa. mod. uk) Martin Dunkley, Airbus (Martin. Dunkley@airbus. com) • CECIM White Paper (CECIM reference model): - and related materials available free-of-charge from Hugh Pickerin (hpickerin@anangle. com) • CECIM information, support, implementation: Hugh Pickerin, Vorticity/Anglian Enterprises (hugh. pickerin@vorticity-systems. com) Dave Lewis, Cost Engineering Solutions (admin@costeng-solutions. com) • Software Engineering Institute (SEI): http: //www. sei. cmu. edu/ • SPICE: http: //www. sqi. gu. edu. au/ © 2004 Vorticity Ltd 40

Acknowledgements The authors wish to acknowledge the prior work performed under the auspices of the Software Engineering Institute at Carnegie Mellon University, and the grant of permission to use the general CMM architecture, utilise the Capability definitions, reproduce Figures and adapt Process Areas 06, 08, 09, 10 and 11 royalty-free (these portions Copyright © 1995 by Carnegie Mellon University). CMM and Capability Maturity Model are service marks of Carnegie Mellon University. The authors wish to acknowledge the assistance freely given by Georges Teologlou (Price Systems) and Galorath Affiliates in promoting CECIM. Finally, the authors would like to thank the members of the EACE CECIM development team (see overleaf) and all survey respondents. COPYRIGHT PERMISSION Permission to reproduce this document for internal use, or to prepare derivative works from this document for internal use, is granted, provided the copyright statements are included with all reproductions and derivative works. © 2002 Hugh Pickerin & Dave Lewis 41

EACE CECIM Contributors E. ADJARI: Astrium (Fr) M. DUNKLEY: Airbus UK G. FILIPPAZZO: Agenzia Spaziale Italiana P. FRAY: Mo. D DPA/PFG D. GREVES: ESA/ESTEC J. HENSON: Westland Helicopters D. LEWIS: Cost Engineering Solutions H. PICKERIN: Anglian Enterprises I. M. TAYLOR: BAES B. SCHREIBER: ESA/ESTEC November 2002 January 2004 42