Independent Evaluation Review Aerospace Medical Equipment Needs AMEN

Independent Evaluation Review Aerospace Medical Equipment Needs (AMEN): Final Investment Decision (FID) Independent Evaluation Review (IER) Team Lead By: Amy Lofstedt Date: May 7, 2010 Federal Aviation Administration

The AMEN program will replace equipment used at the Civil Aerospace Medical Institute (CAMI) in Oklahoma City. This IER highlights the age of the current equipment, the breakdown rate of many of the pieces of equipment, as well as the lack of OEM support going forward. It is a good example of the level of detail required for an ACAT 5 Tech Refresh program, where no benefit analysis is required, no economic analysis beyond the LCCE is required and no Initial Investment Decision necessary. Even though no benefit analysis was required this briefing does discuss qualitative benefits gained from the program. The briefing also highlights the risks associated with the program, utilizing a risk matrix and briefly touching on mitigation strategies for those risks given a medium ranking (it had no high ranked risks. ) This IER does not strictly follow the format of the IER template because it is an ACAT 5 program that required less rigorous analysis than a larger program. However, it does follow the intent of the original template in relaying the necessary information. It is a good example of molding a template to best tell the story of a particular program. The template is not meant to be a fill in the blank instrument. It is a guideline highlighting the pertinent information a good IER must convey. This particular example correctly demonstrates that concept. Federal Aviation Administration 2

Program Overview • Decision Requested – Approval for Final Investment Decision for Aerospace Medical Equipment Needs (AMEN) program • Program Description – The Civil Aerospace Medical Institute (CAMI) in Oklahoma City is the medical certification, education, research and occupational medicine wing of the Office of Aerospace Medicine (AAM). – CAMI’s mission is to improve safety, security, health, and survivability of the human element - aviators, cabin crew and the flying public. – CAMI’s Aerospace Medical Research Division (AAM-600) personnel work in research laboratories and testing facilities with complex scientific, engineering and medical systems. – AMEN technology refresh program was formulated to replace or upgrade a fraction of the research equipment used by CAMI’s Aerospace Medical Research personnel. A brief, one slide description of the proposed program is a helpful tool to familiarize the audience with smaller programs they may not be familiar with. Federal Aviation Administration 3

Business Case Summary • • Problem Statement: Critical, highly technical and specialized laboratory equipment used by aeromedical research personnel in CAMI’s Aerospace Medical Research Division must be replaced or upgraded due to advanced age, lack of support and diminished technology. Problem Statement Supporting Data: – Supports DOT / Federal Aviation Orders, Advisory Circulars, Federal Regulations and Public Laws, including: • • – FAA Office of Accident Investigation (AAI) and NTSB in accident investigation. (CAMI has MOA with NTSB. ) Fire & Cabin Safety and Aeromedical Research Technical Community Research Group’s (TCRG) Priorities (Accident Prevention & Investigation, Crash Survival, Aviation Physiology, etc) Title 14 Code of Federal Regulations, Parts 23, 25, 29, 91, 125, 135 as they relate to Cabin Safety, Biodynamics, Forensic Toxicology, Biochemistry, Functional Genomics, and Environmental Physiology Public Law 100 -591 as it relates to Seats, Berths, Litters, Safety Belts, and Harnesses; Evacuation, Emergency Exits/Placards, Aisle Width, Placards, Safety Equipment and Ventilation as set forth in the Aviation Research Act of 1988 2010 National Aviation Research Plan (NARP) Goals (6) related to Human Protection National Plan for Aeronautics Research and Development and Related Infrastructure – Crash Survival OST Research and Development Budget Priorities – Bioinformatics and Gene Expression research FAA Flight Plan 2009 -2013 (Increased Safety) AAM-600 laboratories are certified compliant with several standards. The use of old and obsolete equipment places these accreditations at risk. Federal Aviation Administration 4

Business Case Summary (Cont’d) • Business Case Summary – Replace or upgrade 121 pieces of laboratory equipment at CAMI • Equipment used in the study of factors that influence human performance in the aerospace environment • Majority of current Bioaeronautical Sciences equipment purchased prior to 2000 (Y 2 K) • 80% of equipment is >xx years old • Equipment has passed useful life and/or is no longer supported by manufacturers • Equipment no longer meets state-of-the-art requirements • Alternative Solution – No alternatives analysis required (ACAT 5 - Tech Refresh / Replacement-in. Kind program) Federal Aviation Administration 5

Business Case Summary (Cont’d) • Program Cost Summary – Baseline Segment FYxx - FYxx AMEN F&E RATY $M $ X. 0 Ops $ X. X Total $ XX. X – Life Cycle Cost Estimate FYxx – FYxx, including all future tech refreshes AMEN F&E RATY $M $ XX. X Ops $ XX. X Total $ XX. X Federal Aviation Administration 6

Business Case Summary (Cont’d) • Program Benefits: –Qualitative shortfalls based on the following identified areas: • Lack of State-of-the-Art Laboratory Equipment - Current lab equipment does not reflect the capabilities offered by advanced technology and procedures currently available in the market. – Utilizing 1980 -1990’s technologies to develop forensic biochemistry procedures (need to keep up with new drugs in market) and assess safety equipment relative to aircraft accident investigation and survivability • Lack of Supportability and Availability - Aging and obsolete lab research equipment is no longer supportable, jeopardizes mission accomplishment and places several laboratory accreditations and standards at risk, including: – American Board of Forensic Toxicologists certification (Subject to audits) – ISO 9000: 2005 and 9001: 2008 certification, Quality Management System (QMS) (Subject to audits) – ISO 14001: 2004 certification, Environmental Management System (Subject to audits) • Lack of Strategic Resource Optimization - CAMI’s critical human resources and their associated research capabilities are being negatively impacted by the use of outdated equipment and associated procedures. – CAMI is the face of the FAA at numerous professional organizations – medical, scientific, and engineering forums – Modern technology needed to support their research activities and resulting products and services – Does not facilitate the recruitment of top professionals Although an ACAT 5 Tech Refresh program does not require a benefit analysis, the inclusion of Qualitative benefits strengthens the case for a positive Final Investment Decision. Federal Aviation Administration 7

Business Case Summary (Cont’d) • Program Schedule – IARD completed April 15, 2010 – FID Approval meeting scheduled for May 24, 2010 – Equipment procurement to take place in FYxx - FYxx • 121 pieces of equipment • Acquisition to involve ~75 vendor transactions – Acquisition Planning/Contractual Document Preparation to commence September 20 xx – First item in-service – March 20 xx – Final In-Service – April 20 xx (xx-month lag between contract award of last significant equipment item and FOC) Federal Aviation Administration 8

Program Description • CAMI has 10 laboratories within its Aerospace Medical Research Division, with focus on 2 critical research areas: – Bioaeronautical Sciences • Forensic toxicology laboratory serves as the primary national research site for toxicology-testing for federal agencies, including the FAA and the NTSB. • Perform research activities regarding pilot certification and performance, aircrew health, atmospheric and radiation risk data, and other factors important to aerospace safety. • Laboratory performs toxicological testing and research on biological materials obtained from living subjects involved in transportation-related accidents. Databases of medical and accident data are maintained. – Protection & Survival • Provide state-of-the-art information, procedures, and equipment evaluations relative to aircraft accident investigation, survivability, health, and security of passengers and crewmembers during normal operations and emergency events. • Acquire autopsies for all fatal aviation accidents and maintain a database for aeromedical research. • Specialized facilities include a hypobaric test chamber, protective breathing equipment and water survival test facilities, a dynamic impact test facility, and aircraft evacuation/aircraft cabin environment simulators (see detail on following slides). Federal Aviation Administration 9

Equipment • • • Majority of 121 automated lab research equipment items is standard COTS equipment that can be purchased from commercial vendors Equipment segregated into 6 categories, based on estimated life cycle Majority of equipment costs relate to the following items: – – • Evacuation and Impact Testing Systems (Aircraft Cabin Evacuation Research Facility and Impact Sled system) – 25 -year life, apprx. $XX million, 2 items discussed in further detail on following page Biochemical Sample Analysis Systems (chromatographs, mass spectrometers, molecular biology instruments and sequencing systems) – 7 -year life, apprx. $X. X million, 27 items Mechanical and Engineering Monitoring Systems (Environmental monitoring, electronic control, anthropometric dummies, calibration systems) – 15 -year life, apprx. $X. X million, 7 items Scientific and Engineering Research Information Systems (Bioinformatics and Radiobiology high performance computing platforms, scientific information system) – 4 -year life, apprx. $X. X million, 23 items Additional categories: – – Biochemical Sample Preparation and Physiological Monitoring Systems (centrifuges, plates, tonometer, oxymeters, extraction tools, balances) - 10 -year life, apprx. $X. X million, 24 items Storage, Cleaning, Machining, and Laboratory Safety Systems - 20 -year life, apprx. $X. X million, 38 items Federal Aviation Administration 10

Equipment - Two Largest Items • 2 largest expenditures comprise xx% of total F&E costs • Average of these 2 pieces of equipment is 37 years old • Aircraft Cabin Evacuation Research Facility (ACEF) – Aircraft cabin fuselage used to conduct aircraft evacuation research. – 43 years old. ACEF is currently immobile and cannot be fully raised, lowered, adjusted for pitch/roll to simulate crashed aircraft. – Narrow-body aircraft evacuation research (including emergency procedures and systems), customer and stakeholder inspections, safety demonstrations, and training activities for cabin crew, TSA, Do. D, Do. J, fire/ law enforcement participation are currently severely limited or eliminated. • High-Impact Accelerator Type Impact Sled System – Simulates impact of high acceleration forces for crash survival testing. – 30 years old. Existing dynamic impact test sled system is able to adequately create the crash forces defined in the existing FAA regulations. However, the predictability (ability to achieve the desired acceleration profile on the first try) and repeatability (the ability to duplicate the same acceleration profile multiple times) of the system is less than can be achieved with modern impact systems Each program has a unique characteristic that can help tell the story. In this case two pieces of equipment served as examples for the entire program. They embodied the age, performance, future OEM support and reliability issues. They also combined to comprise nearly half of the proposed F&E costs. Federal Aviation Administration 11

Equipment – Age and Supportability • Current lab systems average 16 years of age – xx% over 5 years old – xx% over 10 years old – xx% over 15 years old • By 20 xx, xx% of existing equipment will be over 20 years old and unsupportable by vendors. Examples: – Liquid Chromatograph/Mass Spectrometer - vendor has stated will no longer supported in 20 xx ($XXX) – 4 Agilent Gas Chromatograph / Mass Spectrometers will no longer be supported post FYxx - FYxx ($XX - XXX each) – High Performance Liquid Chromatographic System – “breaks Nx a year minimum“ ($XXX) Federal Aviation Administration 12

Equipment – By Lab / Research Facility • Cabin Safety Research - Aircraft Cabin Evacuation Research Facility (ACEF) – • Biodynamics Research - High-Impact Accelerator Type Impact Sled System – • Crash survival testing Forensic Toxicology Research - Gas Chromatograph/Mass Spectrometers – – • Aircraft evacuation research NTSB accident investigations Toxicology and drug effects on humans Environmental Physiology Research - Altitude Chamber Control and Air Handling Systems – – Study of protective breathing equipment for use in emergency situations aboard aircraft. Hypoxia and decompression studies Federal Aviation Administration 13

Program Phasing Schedule/Cost Federal Aviation Administration 14

Costs by WBS Federal Aviation Administration 15

Budget Request Federal Aviation Administration 16

Cost Drivers Cost Driver Total Life-Cycle Cost Estimate F&E Total Cost Initial Acquisition ACEF High-Precision Accelerator Type Impact Sled System Altitude Chamber Air Handling Environment System Bioinformatics High Performance Computing Platform Altitude Chamber Control System All Other Equipment Tech Refresh O&M Total Cost • • • Total BY $M $X $X $X % of Estimate 100 X X X X X Notes WBS 3. 3. 5. 2 – Procurement (including Technology Refreshment) • Cost = $XX. XM (Range: $0 -XX. XM per year) WBS 5. 1. 1 – Preventive Maintenance/Certification • Cost = $X. XM (Range: $X-XX per year) WBS 5. 7. 1 – Program Planning – Cost = $XXM ($XX per year) WBS 5. 10. 9 – Software Licenses – Cost = $XX ($X-XXK per year) WBS 5. 15 – Travel To/From Sites – Cost = $X. X ($XX per year) WBS 6. 5 – Dismantle/Removal – Cost = $X (accrued in FYxx only) Federal Aviation Administration 17

Risk Factors An effective use of a risk matrix quickly shows the factors with mitigation strategies for medium factors. • Medium Risk Factors: – Accuracy of cost estimates: • A 2. 5% additional cost was added to the overall program’s budget to address price risk. • Estimates have been obtained from vendors over the last x years. – Latest quotes were obtained in April 20 xx. – ACEF estimate is based on the combined evaluation of a 1995 study and February 2010 Market Survey efforts. – Biochemistry Equipment Maintenance Cost: • A 3. 33% constant change in preventive maintenance cost was added to biochemistry equipment in FYxx-FYxx. – Based on historical maintenance records which demonstrated a median change in cost of x. xx% over the last 3 -years. – ACEF / Sled Delivery On Time: • Will be monitored by AMEN PM. • Market survey conducted Jan 20 xx, potential vendors have visited existing facility. • Additional schedule slip has been considered – MMAC Contracts Office Capacity / Capability to support tech refresh effort: • Schedule has been vetted by team. • Contractual activities to commence x years in advance of start date. Federal Aviation Administration 18

IP&A Findings A brief synopsis on 4 slides of findings identified by AJF -3 during the analysis. These are broken down by programmatic, cost, benefit and technical findings. A nice summary leading to AJF-3’s recommendation Programmatic Information and Findings: • ACAT 5 Tech Refresh program – Categorization approved by AEB on August 7, 2009 • AMEN bypassed Initial Investment Analysis to go directly to Final Investment Analysis – Tailoring request approved by AEB on September 21, 2009 • No Economic Analysis required – The full risk-adjusted LCCE satisfies the economic analysis requirement • As a non-ATO program, Investment Decision Authorities are Associate Administrator AVS-1 (Margaret Gilligan); CFO (Ramesh Punwani); and FAE (James Washington) – ATO-F is a Review Organization along with AIO • Program is not currently baselined (F&E funds used for Y 2 K purchases in 1999). – Currently using R&D funds to maintain equipment and for new purchases where they can afford. • No Affordability Issues – Funds have been budgeted in FYxx [proposed] process, subject to final approval by EC and JRC on May 25 and June 2, respectively • CAMI maintenance expenses and software licenses currently funded with R&D dollars – In the future, R&D funds that previously went to fund these costs will be freed up for use in research programs (~$XXXXXX per year). • Program is not currently in Ops budget – Program office currently confirming with AVS Ops inclusion in budget starting FYxx Federal Aviation Administration 19

IP&A Findings (Cont’d) Cost Findings: • Largest single ticket items, estimates based on vendor quotes: • ACEF: $X. X • High-Precision Accelerator Type Impact Sled System: $XX. X • Altitude Chamber Air Handling Control and Environmental Systems: $X. X • Bioinformatics High Performance Computing Platform: $X. X • X. X% price risk for all F&E included over and above inflation factors • Estimates based on vendor proposals on equipment • CAMI has been monitoring equipment prices since 2006 • Contracts projected to be fixed-price • Tech refreshes built in to later years in Life Cycle Cost Estimate for future visibility • No plans to add FTEs or CTRs in conjunction with this acquisition Federal Aviation Administration 20

IP&A Findings (Cont’d) Benefits Findings: • Shortfall due to degradation in reliability and maintainability of existing equipment as a result of advanced age and use. – Vendor letters received as support documentation – By 20 xx, XX% of existing equipment will be over 20 years old and unsupported by vendors Federal Aviation Administration 21

IP&A Findings (Cont’d) Technical Findings: • With the exception of a few larger equipment items, COTS laboratory equipment will satisfy the existing shortfall. • The equipment operates autonomously, does not interfere with other NAS systems (e. g. , IT, radar, and telecommunications) and does not connect with the AVS IT architecture • No new software requirements • xx of 121 equipment items will be sole sourced, due to specialization. Program office to solicit multiple vendors where possible and will follow FAA Acquisition Standards. • Largest pieces of equipment (ACEF and Sled, ~ $X million) will be competitively sourced • Acquisition is projected to involve ~xx vendor transactions – Contracts Specialist and 2 Logistics Officers have been identified. • Utilities (e. g. , electrical power, water) will be provided by the MMAC as done today Federal Aviation Administration 22

IP&A Recommendations • Recommend IP&A support Program Office’s request for a FID approval • AVS OPS budget will need to be updated for inclusion of AMEN program • IDAs should be aware that current R&D funds used to support software licenses and equipment maintenance will now be funded under OPS, freeing up $xxx-xxx annually to be used for research activities • Recommend wedge be put in place for FYxx tech refresh • Recommendations are clear and concise and supported by the previous slides. Federal Aviation Administration 23