MDA QSMA Solutions Mr Edward Jopson Assistant Director

MDA QSMA Solutions Mr. Edward Jopson Assistant Director MDA/QSI CQSDI March 27 2007

MDA PROGRAM CHALLENGES KI C 2 BMC GM AB Developmental Systems morphs to Operational System – Reliability standards are inherently more stringent Heritage Programs – Different mindsets, focuses and cultures, integration of these Heritage Programs into a BMDS TH STSS AL 2

MDA PROGRAM CHALLENGES (CONT’D) Spiral Development - Qualification baseline changes and lack of test data to validate models Acquisition Reform - Performance-based specifications and “Best Practices” replacing MIL-STDs 3

MDA SAFETY, QUALITY AND MISSION ASSURANCE DIRECTORATE MDA/QS DESCRIPTION • • • Started October 2002 Currently encompasses 134 personnel in 28 locations Supplier facilities, Test sites, Government facilities Direct Report to MDA Director ROLES AND RESPONSIBILITIES Develop and maintain within MDA a world-class Safety, Quality and Mission Assurance (SQMA) organization Provide BMDS Elements with robust and practical policy, guidance and expertise Develop an integrated BMDS SQMA team across MDA, the program elements, and the Ballistic Missile Defense industry Provide MDA personnel and resources to assist Elements and Industry in SQMA activities Terminate any operation that presents an immediate and unacceptable risk to personnel, property, or mission operations 4

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assistant Director Assurance Integration (QSI) Assistant Director Safety (QSS) Assistant Director Mission Assurance (QSA) Assistant Director Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 5

QS FIELD LOCATIONS MARs, BSOs & SAFETY PERSONNEL ATK, Magna, UT Ft. Greely, AK ABL, Kirtland AFB, NM Ft. Greely, AK Lockheed, Denver, CO JNIC, CO BSOs (6) Eagle Picher, Joplin, MO Raytheon, Woburn, MA Raytheon, Andover, MA Lockheed, Moorestown, NJ Kodiak Facility, AK Aerojet, Sacramento, CA Lockheed, Sunnyvale, CA ATK, Elkton, MD Vandenberg AFB, Lompoc, CA MDA HQ Northrop Grumman, Redondo Beach, CA Boeing, Anaheim, CA Edwards AFB, CA Orbital, Chandler, AZ Spectrum Astro, Gilbert, AZ Raytheon, Tucson, AZ Raytheon, Camden, AR Honeywell, Clearwater, FL KI, Huntsville, AL GMD, Huntsville, AL THAAD, Huntsville, AL GMD/IM & CTF, Huntsville, AL Black = QSL Blue = MAR Green = BSO Red = Safety 6

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 7

MDA ASSURANCE PROVISIONS (MAP) • A standardized set of 14 SQMA provisions, requirements, and processes for mission and safety critical items: - Management - Design and Development - Software and Firmware - Technical and Mission Assurance Reviews - Reliability, Maintainability, and Availability - Parts and Materials Control Program - Integrated Test and Evaluation Program - Test, Measuring and Diagnostic Equipment and Standards - Interface Management - Configuration Management - Control of Nonconforming Items and Materials - Fabrication and Quality - Supplier Management - Safety 8

MDA ASSURANCE PROVISIONS (MAP) (CONT’D) • Provide methods to measure, verify, and validate mission success through: - Collection and analysis of metrics - Risk Assessment, technical evaluations, independent assessment, reviews - Conducting tests • Promote continuous process improvement to reduce cost, improve productivity, and assure SQMA • Specific applicability is defined in Mission Assurance Implementation Plan (MAIP) – Verification Matrix MAP is on/almost on all MDA contracts 9

MDA ASSURANCE PROVISIONS (MAP) COOPERATIVE APPROACH IMPLEMENTATION Near-Term MDA Assurance Provisions Mid-Term Evaluate / Understand Element MA Practices Long-Term Improve Element MA Programs Build Integrated MA Team Throughout MDA Define / Implement Mission Assurance Program For The BMDS Refine And Improve MAP And MDA MA Processes Mandate Use Of “Golden Round” Philosophy – Test Like You Operate Flow Down MA Requirements To Contractors And Vendors Formally Incentivize Mission Success MAP As A Core Standard On All New Contracts Perform Independent Reviews Matrix QS Personnel Into Elements • Perform Gap Analysis • Collect / Share / Apply Lessons Learned • Prioritize High Risk Items Independent Assessment • QS Personnel Matrixed Throughout MDA Elements • Provide Comprehensive “Second Look” 10

MISSION ASSURANCE IMPLEMENTATION Design And Development Manufacturing Design & Development Element Integration & Ground Test Flight Testing & Verification Element Increasing Opportunity Integration & Manufacturing For Block 06 Ground Test Element Maximum Opportunity Integration & Manufacturing For Block 08 Test Ground Flight Testing & Verification IDO Block 06 Block 08 Time 11

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 12

MDA ASSURANCE REPRESENTATIVES MARs provide MDA with deployed Mission Assurance resources for direct and unfiltered supplier insight, with the leverage to effect value added change and mitigate risk in a timely fashion, and serve as MDA’s Mission Assurance ambassadors to its Supplier community, associated agencies and organizations Responsibilities: - Rapidly identify, report and resolve technical issues on both hardware and software that impact the safety and reliability of the Missile Defense program - Immediately report to the supplier’s program office and to MDA/QS any safety concerns/issues - Immediately report to MDA/QS significant hardware/software issues impacting upcoming missions - Report and resolve safety, quality, and reliability issues/concerns impacting MDA hardware and software 13

MDA ASSURANCE REPRESENTATIVES (CONT’D) • Responsibilities (cont’d): - Represent QS Directorate and Cognizant Program Office Provides DCMA and contractor direct access to MDA Provides MDA Director unfiltered insight into program operations in real-time Participates in design integration, test, qualification activities and issue resolution - Ensures that all applicable safety, quality and mission assurance provisions are being followed - Disseminates lessons learned to counterparts at all BMDS components and DCMA - On-site authority for QS LOD to DCMA 14

MAR vs DCMA ROLES MARs Mission Assurance DCMA Quality Adherence Dedicated MDA Resource (Agency Focus) Matrixed, Non-Dedicated, Contract Focus MA Ambassador (Leadership & Influence) N/A Direct Feedback Non-Direct Feedback Pre-Emptive Role (Initiate Risks) N/A BMDS Perspective Facility/Product Specific System & Subsystem Cognizance N/A System & Subsystem Test Adequacy Test Surveillance Adequacy Of Requirements Requirement Compliance Acceptability Of Issue Resolution Surveillance Adequacy of Quality Process Mandatory Inspections Adequacy Of Process & Procedure Adherence Cross Element Comm/Coord N/A 15

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 16

Unfettered Access Direction 17

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 18

MDA ASSURANCE AUDITS AUDIT PROCESS • Product Design • Product Requirements • Processes “Comparison” • SQMA Requirements • MDA/Industry Best. Practices (MDA Assurance Provisions • Contractor Plans, Procedures, Practices and Records & Other MIL-STDs/Guidelines) 19 19

MDA ASSURANCE AUDITS • Validate that end items at a selected level conform to engineering design requirements • Validate end item at a selected level is qualified • Compare fabrication methods and acceptance test levels against MDA & industry standards as appropriate for MDA use • Validate production readiness • Assess Contractor/Government Mission Assurance practices, procedures, and implementation 20

MISSION ASSURANCE AUDIT SCOPE • • • Management Systems System Design Integrated Test and Evaluation Supplier Management Configuration Management Safety Reliability, Maintainability, and Availability Software Parts, Materials, and Processes Fabrication and Quality Management System – AS 9100 Other Safety, Quality, & Mission Assurance Practices 21

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 22

MDA ASSURANCE GROUP The MDA Assurance Group (MAG) is used to ensure industrywide understanding and implementation of the MDA Quality, Mission Assurance and Safety requirements, as set forth by the MDA/QS Director MAG membership includes key Government and aerospace industry executives knowledgeable of quality enterprise strategic planning, Quality Management System (QMS) requirements / implementation, mission assurance principles, and knowledge of their companies Missile Defense programs 23

MDA ASSURANCE GROUP (CONT’D) • Primary responsibilities of the MAG are to: - Review and discuss the MDA Quality, Mission Assurance and Safety policies and strategies set forth by the MDA Director to make certain implementation facilitates mission success - Identify, discuss and resolve significant, cross-cutting systemic issues affecting MDA programs - Understand the activities of other Aerospace Industry groups/associations like SQIC, AAGQ, AIA, etc with purpose to eliminate redundant activity and to gather their lessons learned • MAG Meetings were held on 15 March 2005 and 28 February 2006 • Key MAG Members: MDA, NASA, Boeing, Aerojet, Northrop Grumman, Raytheon, Lockheed Martin, ATK • Ex: Spawned “Stakeholder Initiatives” 24

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 25

MDA ASSURANCE ADVISORIES • MDA Assurance Advisories (MAA) provide information about problems, potential problems and failure experience data on parts, components, materials, manufacturing processes, specifications, computer software, test equipment and safety • Advisories are exchanged to improve quality, reliability, and delivery schedules and to reduce costs • Implemented in October 2003, QS has published 23 Advisories. Distribution to the MDA program elements, Industry, NASA, NRO and SMC 26

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 27

DCMA DELEGATION OF AUTHORITY • Agreement between the Missile Defense Agency Director and the Defense Contract Management Agency Director… - Establishes quality oversight requirements for mission critical items - Defines MDA expectations and priorities - Empowers DCMA to provide hands on quality insight/oversight - Establishes quality assurance insight/oversight requirements throughout the supply chain - Provides commonality and consistency for quality assurance implementation - Provides a government influence on mission success 28

DELEGATION PROVIDES… • DCMA Quality Assurance oversight requirements for: - Quality assurance surveillance strategy & plan - Process surveillance activities - Contract management offices reporting and notification - Material review board activities - Mandatory Government Inspections (MGIs): Ø Critical lifts and operations Ø Test setup (i. e. , equipment and crew certifications, documentation verification) Ø Final inspection Ø Pre-shipment and shipment activities (i. e. , black box closeout, DD 250, etc. ) 29

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 30

PARTS, MATERIALS AND PROCESSES PROGRAM • State of MDA PMP Today - Non-standardized PMP management - COTS Technology and Obsolescence management lacks definition - MDA lacks a comprehensive product engineering process • MDA/QS PMP - MDA-wide PMP Management Plan - MDA Parts, Materials and Process Board/Control Boards - MDA Preferred Parts List - Pilot Program developed for GMD Product Lines 31

Why the PMAP? • PMAP meets two of the Director’s goals: - Support the operations and sustainment of capabilities fielded to the warfighter – Eliminates variability in screening and qualification of PMP – Improves the probability of mission success – Ensures repeatability in the qualification baseline - Develop an integrated capability for Block 2006 and beyond – Through a fully integrated Agency PMP program Uniformity of Requirements Across All Elements 32

Why the PMAP is Necessary… Industry State of Affairs • Overseas Plants - • Unauthorized Distributors - • Limited processes and material change knowledge Military parts built on comm. manufacturing lines Easy access: foreign countries reverse engineer military technology No established relationship with part vendors, poor quality, no recourse – traceability issues Inconsistency in vendor lot to lot parameters Counterfeit fasteners - rivets, nuts, bolts, washers Lead-free Initiative - Pandemic for Hi-Rel systems No accepted method for tin-whisker mitigation Government - military and space application • Obsolete Parts • EEE parts represent decreasing market share Commercial Driven Industry = Greater Controls Required 33

Why a PMP Program is Necessary… Counterfeit Components • Unauthorized Manufacturers - Build fake parts, selling cheap or defective devices at brand name prices Counterfeit Genuine - Buy low-power die for high-power packages - Parts marked with “high-rel” part no. - Fabricate parts to substandard processes • China # 1 Seller Counterfeit capacitors with substandard electrolyte - $1 - 10 billion per year (estimate) - 35% of world market by 2010 Counterfeit Genuine - Fab sites expanding rapidly, little govt control - Patent infringement • Companies Counterfeited - Motorola - On Semi - Toshiba - National - PMC-Sierra Note the difference in part numbers 34

PMAP Requirements & Section Highlights • Introduction Scope - Purpose • Applicable Documents Government, Commercial • PMP Management Structure, Roles, and Responsibilities PMP Control Board • PMP Requirements Program categories and definitions Management of PMP selection & Quality Requirements Applicability Matrix PMP Procurement Rad Hard, GFE, GFM PMP Qualification Non-Conforming Materials and Parts Failure Analysis ESD Control, Environmental Controls, Handling Controls Preservation, Packaging and Storage Requirements • Appendices Derating PEMS COTS Radiation Hardness Prohibited Parts & Materials 35

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDa Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 36

SUPPLIER ROADMAP • Supplier Roadmap and Supply Chain Database - 453 Suppliers in 39 states and 2 foreign countries • Can be searched by: - Congressional District - State - Supplier - Program 37

SUPPLIER ROADMAP (CONT’D) 38

MDA/QS Director Quality, Safety And Mission Assurance (QS) Assurance Integration (QSI) Safety (QSS) Mission Assurance (QSA) Quality (QSQ) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Govt/Industry Lessons Learned Program MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap 39

MDA LESSONS LEARNED ROADSHOW “The frustrating part about it is we have not had an incident that, in my view, can be classified as a failure of the technology. . . Someone didn’t perform a process right, somebody left out a piece of material where they shouldn’t. . . ” Former MDA Director, Lt Gen Ronald Kadish, December 2003 Roadshow Targets all audiences: Program Offices, President, Vice President, Director, Chief Engineer, Systems Engineering Manager, IPT Leads, Engineers, Quality, Safety and Mission Assurance Personnel and Technicians Discusses Lessons Learned on IFT-13 c and IFT-14 Tests 40

Why Quality, Safety and Mission Assurance? Test 1 Test 2 Test 3 Root Cause EKV failed to separate due to an ASIC lead fatigue failure in the laser firing unit on the EKV Fundamental Issue Inadequate laser firing unit qualification test. Stiffening foam was removed from a board. The random vibration test was conducted without power-on so the problem could not be detected on the ground. Countdown automatically aborted due to a built- Roundoff error propagation caused a timing in-test anomaly that detected excessive drops of mismatch. When the SIGI and FC clocks crossover, timing jitter occurs and occasionally Electronic Control Unit (ECU) messages data is sent to the ECU when it is busy, resulting in no responses. To avoid a recurrence, need to raise the tolerance level for missed messages -- changing one line of computer software code. One of the last steps in the launch sequence is to open the silo doors and retract the horizontal stabilizers that hold the missile in place. Sensors indicated that one of the three lateral support group mechanisms had not retracted, causing the missile to abort launch. Corrosion and dissimilar metals (galvanic corrosion) caused the failure. One mechanism failed to latch due to wrong crush blocks. 41

MDA LESSONS LEARNED ROADSHOW (CONT’D) Company Group Location Northrop Grumman Lockheed Martin Raytheon THAAD Lockheed Martin AF SPO Raytheon Spectrum Astro Northrop Grumman Aerojet KEI MA Leads Aegis RMS IDS Program Office THAAD & MKV THAAD BV+ Courtland ABL RMS NFIRE, STSS MDA Fair Lakes, VA Reston, VA Moorestown, NJ Woburn, MA Tewksbury, MA Huntsville, AL Sunnyvale, CA Huntsville, AL Elkton, MD Albuquerque, NM Tucson, AZ Gilbert, AZ Los Angeles, CA Sacramento, CA 42

WHAT MDA HAS DONE TO INCREASE SQMA • Increasing Quality Assurance rigor by: - Organizing for Mission Success - Changing Culture - Onsite MDA Mission Assurance Representatives (MARS) - MDA Assurance Provisions (MAP) & PMAP - Defense Contract Management Agency (DCMA) Delegation of Authority Letter - System and Range Safety Oversight - Safety and Occupational Health Getting Back to the Basics 43

GOING FORWARD • Change Customer and Contractor Cultures - Corporate Leadership and UNIVERSAL, CONTINUING Commitment • Increase Process Discipline – Supplier Management - Management and Engineering Process Rigor - Establish and Maintain Basic Set of SYSTEM Standards (MDA MAP – Core Standards) - Ensure basics are CONTINUOUSLY accomplished by BOTH Customer and ALL Supplier Tiers - Ensure Systems Engineering, Disciplined Test Management, Quality Management, Risk Management, Configuration Management • Organize for Mission Success - Clear, continuing organizational accountability - Robust engineering organization - Independent, capable SQMA organization and personnel 44