Cost Analysis for the NATO ALTBMD Feasibility Study

Cost Analysis for the NATO ALTBMD Feasibility Study M. C. Smit (TNO-FEL, The Netherlands) D. J. Brown (MOD DPA PFG, UK) SCEA conference June 15 th - 18 th, 2004 Manhattan Beach Marriott Los Angeles, California, USA

Contents Introduction ALTBMD Feasibility study Introduction Cost Analysis Team (CAT) Tasks and activities Challenges Results Lessons learned SCEA conference June 2004 CAT lessons learnt ALTBMD FS 2

Introduction ALTBMD: Active Layered Theatre Ballistic Missile Defence Main objective: “Protection for deployed NATO-forces” Defence against Theatre Ballistic Missiles To provide an integrated NATO “systems of systems” based on National components Layered to give very low “leakage” rate – defence in depth: lower layer (e. g. Patriot), upper layer (e. g. THAAD) SCEA conference June 2004 CAT lessons learnt ALTBMD FS 3

SCEA conference June 2004 CAT lessons learnt ALTBMD FS 4

SCEA conference June 2004 CAT lessons learnt ALTBMD FS 5

Introduction ALTBMD Feasibility Study Two contractor teams headed by Lockheed Martin (JANUS-team) and Science Applications International Corporation (SAIC) Principle objectives: Analysis on the feasibility of meeting the requirements set out in NATO Staff Targets (system requirements) Recommend the architectural (systems of systems) solution that achieves a balance among performance, time schedule, costs and risks Task description in Statement of Work (SOW) with EEAs and MOEs. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 6

Example EEAs and MOEs SCEA conference June 2004 CAT lessons learnt ALTBMD FS 7

NATO- Organisation ALTBMD FS NAC North Atlantic Council CNAD Conference of National Armaments Directors TMD PG Missile Defence Project Group NATO ALTBMD Feasibility Study SMT Study Management Team • Defend NATO Deployed Forces • 2001 - 2003 • NST NSR • IOC 2010 SAIC consortium SSG Study Support Group JANUS consortium CAT Cost Analysis Team NL, US, GE, UK, IT MST Modeling & Sim. Team NL, US, FR, GE, UK, TU SCEA conference June 2004 CAT lessons learnt ALTBMD FS 8

Contractor Teams ALTBMD FS SAIC Boeing (US) Lockheed Martin (US) MBDA (UK, FR & IT) EADS LV (FR) TRW (US) Qineti. Q (UK) BAE Systems (UK) Diehl (GE) EADS LFK (GE) TNO (NL) IABG (GE) SCEA conference June 2004 CAT lessons learnt ALTBMD FS 9

Feasibility Study Timeline NST PAPS Feasibility Phase July 99 1999 2000 2001 Preparation Phase C N A D T M D P G 2002 Study Phase Study Preparation • Data Packs • IFB Prep • NC 3 A/SMT Manning 2003 NSR 2004 Consolidation Phase SAIC Team Evaluate & Consolidate Team Janus (NC 3 A) Draft NSR + Docs (TMD PG) Contract Award SCEA conference June 2004 CAT lessons learnt ALTBMD FS 10

Introduction CAT Established in April 2001 Participating nations: NL, US, UK, GE. DK withdrew, FR participated incidentally, IT joined in January 2004. General tasks CAT: – Support TMD PG and Study Support Group (SSG) on Cost issues – Support NATO Studies Management Team (SMT) – Report progress activities at SSG-meetings. – Comment on results of contractor teams – Develop independent Cost estimate SCEA conference June 2004 CAT lessons learnt ALTBMD FS 11

Preparation Phase (until mid 2001)

Tasks and Activities Preparation Phase Produce Cost Breakdown Structure (CBS) and Cost Format to be used by Contractor Teams Definition of Cost Elements as part of CBS (based on OSD CAIG) Provide general ground rules and assumptions to be used and constraints to be taken into account by Contractor Teams Decide on cost estimating approach by CAT – Agree on methods, techniques and tools to be used. Assign responsibilities to participating nations, decide on work share SCEA conference June 2004 CAT lessons learnt ALTBMD FS 13

Challenges in Preparation Phase Models from earlier studies could not be used. No existing cost model that satisfied all needs --> new models(s) need(s) to be developed. No relevant cost related data in data packs No direct contacts possible with Cost Experts of Contractor Teams, to agree on approach, models, data to be used Only indirect contacts via Study Management Team. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 14

Results of Preparation Phase Document with Cost Format and Ground rules and Assumptions. Requirement for following CAT models: – – – Interoperability (mainly BMC 3 I) (US), System development and production (UK-GE), Operation and Support Cost (US-NL), System level spreadsheet(s) (UK) Architecture level spreadsheet (NL-UK) All data and assumptions are recorded in Master Data and Assumptions List (MDAL) SCEA conference June 2004 CAT lessons learnt ALTBMD FS 15

Study Phase (mid 2001 - early 2003)

Task and Activities Study Phase Development spreadsheet models to be used for independent cost estimate CAT Gather cost related information on systems, potentially part of architecture (continuing effort) Independent Cost estimate of potential ALTBMD system configurations and architecture Evaluate interim and final reports contractor teams SCEA conference June 2004 CAT lessons learnt ALTBMD FS 17

Challenges in Study Phase Referring to independent cost estimate Lack of data!! Some nations contributing systems were not involved in CAT At that time: No agreed composition of systems No generic architecture available Delivery of contractor databases to CAT was difficult SCEA conference June 2004 CAT lessons learnt ALTBMD FS 18

Results of Study Phase Cost information mainly gathered from limited sources. For non-US/UK systems assumptions had to be made Findings evaluation reports contractor teams: – Most EEAs/MOEs were answered sufficiently – No general overview and no systematic set-up of cost information in reports (Cost Volume is missing) – Contractors used different set-up of systems – Traceability (references, assumptions, clarification, calculations) results were hard to find in reports – Not possible to perform excursions with results contractor teams by changing inputs (only numbers, no process) SCEA conference June 2004 CAT lessons learnt ALTBMD FS 19

General Overview Model Requirement NATO CAT Industry Operation & Support System Interoperability Government Production & Development Architecture Interoperability Other Modelling Data Scenarios Assumptions SCEA conference June 2004 CAT lessons learnt ALTBMD FS Architecture Cost Forecast 20

Common Issues Classification Geographically diverse membership Commercially sensitive Different architectures for different scenarios Need to identify estimates of – cost to NATO – contributed value by Nations SCEA conference June 2004 CAT lessons learnt ALTBMD FS 21

Cost Modelling Structure Operation & Support Inter-Operability Componen System t System Component tt Common Data Architecture Summary System Configurations Common Data SCEA conference June 2004 CAT lessons learnt ALTBMD FS 22

Component/System Cost Model Linked Excel-spreadsheets First version realised in study phase, further refined in consolidation phase. Combines estimates for individual systems • development and production • operation & support Building blocks used for components – sensors – launchers – interceptors Allows numbers of building blocks to vary SCEA conference June 2004 CAT lessons learnt ALTBMD FS 23

Component/System Cost Model (cont’d) ‘Three point’ estimating based on @RISK simulation tool Problem: Different generic Types of component/system Different Source countries Lack of verified data (audit trail) System Configurations were variable UPC used in Operation & Support Cost CER System level interoperability hardware cost Common Cost elements Estimates based on own assumptions and available data Operation and support costs integrated at system level. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 24

Example Component/System Output Numbers are only given to illustrate SCEA conference June 2004 CAT lessons learnt ALTBMD FS 25

Operation & Support Cost Model Excel-spreadsheet, based on OSD CAIG cost elements Data obtained from open sources Problems: Some components/systems are in early stages of development. Some component/systems are multi-role. Different Source countries Lack of verified data (audit trail) UPC based CERs for some aspects Operation and support costs incorporated at system level. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 26

Interoperability Model Software intensive systems integration into NATO C 2 Excel-spreadsheet Problem: no details on costs available Estimates required numerous assumptions Main effort: SLOC count SCEA conference June 2004 CAT lessons learnt ALTBMD FS 27

Description Arch Level Spreadsheet Not realised during study phase, but in consolidation phase Combines estimates for – interoperability – individual systems • development and production • operation & support Allows numbers of systems in architecture to vary Makes a distinction between – cost to NATO – contributed value by Nations SCEA conference June 2004 CAT lessons learnt ALTBMD FS 28

Example Output Architecture Costs Numbers are only given to illustrate SCEA conference June 2004 CAT lessons learnt ALTBMD FS 29

Consolidation Phase (early 2003 - early 2004)

Tasks and Activities Consolidation Phase SMT: to develop consolidated architecture from contractor teams results Calculate costs recommended systems Calculate costs recommended architecture(s) Calculate costs for excursions on architectures Contribution to recommended architecture report SCEA conference June 2004 CAT lessons learnt ALTBMD FS 31

Challenges in Consolidation Phase Rationalising results Contractor teams difficult: – Both Contractors and CAT used different compositions of systems – No ability to change inputs in Contractor models Tasks of Working groups were performed in parallel. – System composition became available late – Architectures subject to many changes Composition of systems and architectures caused changes in CAT systems and architecture models SCEA conference June 2004 CAT lessons learnt ALTBMD FS 32

Results of Consolidation Phase Cost estimates from Contractor teams were of limited use CAT provided input to recommended architecture report (answering EEA/MOE table 21) CAT Cost results make a distinction between – Cost to NATO – Value of systems contributed by Nations SCEA conference June 2004 CAT lessons learnt ALTBMD FS 33

Final Results Cost analysis report – – description of all CAT activities in ALTBMD FS provision of results cost model descriptions description of data used and assumptions made (MDAL) SCEA conference June 2004 CAT lessons learnt ALTBMD FS 34

Lessons Learned Ask contractor for separate Cost Volume as a deliverable with a systematic set-up of cost information To have discussions with cost experts Contractor Team Closer co-operation with cost experts Contractor Team Ask specifically for calculations and formulas in spreadsheets to be provided (allows for excursions through changes in inputs). Ask for MDAL or CARD that summarises all data, assumptions and formulas used. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 35

Follow-on activities: Missile Defence study New FS (Missile Defence study) started early 2004 Initiated at NATO summit in Prague Main objectives: Examine options for protecting NATO territory, forces and population centres against the full range of missile threats Recommend options and configurations for system elements, including sensors and Command & Control Determine the best mix of systems and capabilities to obtain a NATO missile defence architecture to meet MOR, considering performance, cost and risk SCEA conference June 2004 CAT lessons learnt ALTBMD FS 36

Lessons Learned Taken Into Account Closer contact with cost estimator contractor team but not a common effort Meetings with cost estimator cost team: First meeting already held to discuss way ahead: Common understanding of approach, ground rules and assumptions Contractor agreed to produce separate Cost Volume All assumptions are recorded in an MDAL SCEA conference June 2004 CAT lessons learnt ALTBMD FS 37

Health Warning Please note: The contents of this presentation provide the views of the two authors and should not be taken as the official views of NATO, UK Mo. D or NL Mo. D. SCEA conference June 2004 CAT lessons learnt ALTBMD FS 38

I need HOW Questions ? SCEA conference June 2004 CAT lessons learnt ALTBMD FS this! Could you. . . ? ? WHEN? 39