Commercial Aviation Accidents Before and During Alaska Capstone

Commercial Aviation Accidents Before and During Alaska Capstone (ADS-B, FIS-B, Terrain Awareness, IFR Infrastructure) • Fifth USA/EUROPE Seminar on ATM R&D • 24 June 2003 • Worth Kirkman © The MITRE Corporation This is the copyright work of The MITRE Corporation and was produced for the U. S. Government under Contract Number DTFA 01 -01 -C-00001 and is subject to Federal Aviation Administration Acquisition Management System Clause 3. 5 -13, Rights in Data-General, Alt. III and Alt. IV (Oct. 1996). No other use other than that granted to the U. S. Government, or to those acting on behalf of the U. S. Government, under that Clause is authorized without the express written permission of The MITRE Corporation. For further information, please contact The MITRE Corporation, Contracts Office, 7515 Colshire Drive, Mc. Lean, VA 22102, (703) 883 -6000

6/24/03 Capstone • Joint initiative by FAA Alaska region and industry to improve safety and efficiency of commercial aviation • ADS-B and related broadcast technologies – – Traffic and terrain situational awareness Flight information including weather in the cockpit Surveillance and radar-like ATC services Additional automated weather sites and GPS instrument approaches • Capstone Phase 1: ground systems and avionics for commuter and air taxi aircraft (FAR part 135) in isolated southwest Alaska – Near Yukon and Kuskokwim rivers – the “YK delta” 2 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Aviation in the YK Delta • Usually the only means of transport • Limited services and infrastructure – Unpaved airstrips – Mostly non-radar – Few weather sites • Aircraft: single-engine or light twin • Operations: mostly VFR • Commercial accident rate 2. 5 times the remainder of US 3 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Safety Problems Targeted by Capstone • Enroute controlled flight into terrain (CFIT) • Mid-air collisions • Inadequate flight information – Especially weather • Inadequate support for IFR operations 4 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Overview of Capstone Capabilities 5 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Capstone’s Impact on Accidents: Overview of Approach • Characterize accidents before and during Capstone • Model the projected impact of Capstone – Different accidents types and causes – Assessed affect on historical accidents if Capstone had been in place – Progress toward putting Capstone in place – Changes is operations • Compare projections against (and learn from) accidents that still occur • Characterize significance of / confidence in any observed changes 6 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Categorizing Accidents • Analyze detailed narratives from the National Transportation Safety Board (NTSB) for each accident • Apply a decision tree to categorize each accident – Based upon causal / contributing factors and phase of flight – Categories specialized toward understanding Capstone’s effectiveness • Assess likelihood of Capstone preventing each historical accident – Rules-of-thumb for sub-categories based on expert judgment – Adjust for specifics of each accident scenario that would limit (or increase) effectiveness 7 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Accidents and Fatal Accidents Before Capstone YK Delta / Part-135 / 1990 -1999

6/24/03 Accident Categories Affected by Capstone, Potential Effectiveness if Fully Implemented • Navigation – Enroute CFIT, most often in reduced visibility or ‘Flat Light” (Capstone potentially ~90% effective) – Terrain Clearance Floor (TCF) violation on approach/depart (Capstone Phase 1 not effective) – Mis-location of aircraft (moving Map potentially ~100% eff. ) • Traffic – Mid-Air collisions (Capstone potentially ~100% effective) • Flight Information – Inadequate Weather data, especially icing, but also visibility* (Capstone effect expected but not quantifiable from scenarios) *Surface winds contributing to take-off and landing accidents are included in take-off and landing categories 9 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Anticipating Accident Rates With Capstone Formulas for Projections - Overall and by Category • Project numbers of accidents: Historical accidents X growth in operations • Project reductions in navigation and traffic accidents from: X X % of Aircraft operations equipped % of Other aircraft equipped (affects traffic only) % of Pilots adequately trained % of Historical accidents in preventable sub-categories Assessed effectiveness for historical accident narratives • Weather and other categories should benefit, but effectiveness could not be quantified from historical narratives 10 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 2000 -2001 Progress on Implementation: Equipage and Pilot Training Operations: 50% Aircraft Days:

6/24/03 Anticipated vs. Observed Accidents YK Delta / Part-135 / 2000 -2001 12 ©

6/24/03 Assessing Significance and Confidence • Accident counts in all categories are close to two-year standard deviations from previous years – Not statistically significant • Further analyses: Compare accidents by Capstoneequipped vs. non-equipped aircraft – Distribution of accident types – Rate of total accidents per number of operations 13 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Accidents by Equipped and Non-Equipped Aircraft YK Delta / Part-135 / 2000 -2001

6/24/03 Comparing Accident Rates • Assess separation and stability of cumulative accident rates over time • Challenge: poor data on operations Ö Ö Ö ~ – – Operations at Bethel Enplanements and fleet changes at Bethel Capstone equipped / non-equipped ratio at Bethel (SVFR) Aviation fuel usage Operations and enplanements at villages Airframe hours of operation • Seasonal (monthly) operations at Bethel averaged between 2000 & 2001; equipage ratios from SVFR 15 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 16 Non-Equipped vs. Capstone Equipped Accident Rates YK Delta / Part-135 / 2000

6/24/03 Interim Assessments & Planned Extensions to the Analyses • 2000 -2001 is too soon for statistically significant changes in specific types of accidents or to accidents over-all - though indications appear “promising” • 2002+ will provide longer basis for comparison and larger samples of accidents • Plan to obtain better data on operations – Operations counts / miles / hours from operator records – Gather reported operations changes associated with Capstone that should guide additional data collection/analysis • Examine trends – vs. nearby non-Capstone areas – vs. all of Alaska 17 © 2003 The MITRE Corporation. All Rights Reserved.

6/24/03 Additional Details and Updates • The Safety Impact of Capstone Phase 1 – An Interim Assessment of 2000 -2001 (MPW 0000150 Revision 2) – http: //www. alaska. faa. gov/capstone/docs/mitre%20 study. pdf • Updated analysis is in progress – Implementation Progress and Accidents reported through 2002 – Watch http: //www. alaska. faa. gov/capstone 18 © 2003 The MITRE Corporation. All Rights Reserved.