Heli Stat An Automated Risk Assessment Tool for

Heli. Stat™ An Automated Risk Assessment Tool for Helicopter Safety Management Systems Alex Richman Algo. Plus Consulting Limited Halifax NS Canada Arichman@algoplusaviation. com Leonard Mac. Lean Dalhousie University Halifax NS Canada T SEPT 28 Copyright© 2005 Algo. Plus Consulting

Abstract Safety Management System (SMS) developers have challenges in: Becoming proactive Assessing risks systematically Prioritizing This presentation describes Heli. Stat™, a computerized system which automatically performs these functions for mechanical malfunctions (SDR and MMIR) and operational incidents (CADORS)

The Evolution of Safety Culture (and Safety Management Systems) 5. GENERATIVE Systematic knowledge about operation of system as a whole. safety is how we do business round here 4. PROACTIVE we work on the problems that we still find 3. CALCULATIVE we have systems in place to manage all hazards 2. REACTIVE Safety is important, we do a lot every time we have an accident 1. PATHOLOGICAL who cares as long as we’re not caught Increasing vigilance for potential hazards Adapted from Hudson , Leiden Univ, 2004

SMS requires Data

SMS Implementation There are multiple approaches to SMS Transport Canada’s Guide is more detailed than others in specifying the process (TP 14343) Phase 1: Plans and procedures Phase 2: Reactive system (3. 1) Capture internal data Monitor/analyze trends Phase 3 - Proactive hazard identification from external sources(3. 2) Risk assessment (3. 4)

Advantages of Proactive Analysis 1. Because of their larger size, systemwide databases are far more likely to yield information about events which might occur later in a smaller fleet 2. Larger data bases are more likely to reveal trends 3. Benchmark comparisons are possible

Reactive/ Proactive Dichotomy Reactive §Post facto §“Tombstone mentality” §Investigation of event §Microscopic §“Smoking gun” §Forensic cause Proactive §Identify potential hazards §Wariness § General analysis of system §Macroscopic §Chain of risk factors §Probabilistic Richman & Murphy 2005

Sources for Proactively Identifying Potential Hazards • Company experience: reports, minutes • Managers’ perceptions • Workforce opinions • Audit reports • Previous hazard analyses • Hazards identified by other organizations Safety and reliability data recording systems: CADORS, BASIS SDR, MMIR TC TP 1381 E

SDR and MMIR Reports: Mechanical and Maintenance Service Difficulty Reports: • Federal Aviation Administration and Transport Canada • Mandated • “Mechanical problems, defects, malfunctions affecting safety” Mechanical Malfunction Information Reports: • Helicopter Association International • Voluntary • Broad definition Web sites are useful for locating specific reports but are less useful for risk assessment and trend assessment

Questions about Safety. Significant Events (SSE) • Are there significant increases over time? • Are SSEs “consistent” over time (SPC)? • Which SSEs are more severe and more likely to occur (risk assessment)? • Are SSEs more likely in one model or operator? • Are there historically unusual SSEs? • Which SSEs should have more priority for further attention? • How can we identify SSE precursors ?

GAIN Initial Survey of 15 Safety Officers – Too much time spent on routine tasks – Little time for proactive safety efforts – Follow-up is not always well tracked – Need outside benchmarking – Need better analytical tools – Clear safety strategy infrequent (1/15)

Problems in SMS Analyses • Complexity of data preparation and updating • Difficulties in integrating data and software into IT infrastructure • Need for computer proficiency • Acquiring analytic skills • Lack of time

Lots to Do: Too Little Analysis

SMS Software Requirements • Easy to use • Contain proactive data bases • Identify hazards and assess risks (Likelihood, severity, trends) • Indicate priorities • Generate documentation • Monitor system continually

Continuous Monitoring and Automated Analysis • (Morningstar is a financial example) – Continuous monitoring of updated databases – Near-real-time reporting of updates to benchmarks, trends, rankings, priorities – – “Pushes” analytic results to user – Very user-friendly for SMEs Heli. Stat™ uses this model •

What is Heli. Stat™? A subscription service for web-based analyses of mechanical malfunctions reported to the FAA- Service Difficulty Reports and to Helicopter Association International’s Mechanical Malfunction Information Reports

Heli. Stat™ is Designed for Aviation Specialists Without Data Analytic Skills TM

Heli. Stat™ • • Incorporates multiple databases SDR/MMIR Near-real-time data Uses census denominators Compares models/operators with industry wide benchmarks Model specific Source data accompanies analyses Prioritizes problems Shares and tracks reports

Heli. Stat™ is a System 1 E-mail list for model by JASC 2 Threat Matrix 3 Source report 4 Share Report 5 Out mail list

Priority ALERTS in E-mail A B C D A. Risk assessment traffic light summarizes Threat Matrix B. Prioritized C. Recent reports, past 5 weeks D. Reports not yet viewed

Systematically Defined Risk Matrix Significantly above average Relative Likelihood Below average Minimal Hazard Increased

Reliable and Objective Risk Matrix Calculation: • Relative likelihood of occurrence: Rate in the specified model Benchmark rate for all the other models • Hazard is derived from the FAA Severity score for specific ATA/JASC codes and the Flight Safety Foundation’s CARE project for classifying “Nature of Condition” and “Precautionary Procedures” in SDRs

3 Graphs Combined to Show Time Trends

Time Trends in One Risk Assessment Matrix

Risk Matrix Summarized in Traffic Lights • Relative likelihood • Hazard level • Time Trends • Recent Increase

Risk Matrices for Individual Parts • No limit on number of parts

Individual Narrative Reports for 6 Months

Historically Unusual Events: Unique, First, Rare, Cluster

Heli. Stat™ Overcomes Many Pitfalls in Risk Assessment • • Objective, quantitative assessment Specified time span Indicates trends over time - trajectory States nature of the hazard Shows source narratives Automated without specialist assessment Sensitive to short-term changes Continuous monitoring

Heli. Stat™ System for Sharing Reports The user can e-mail the JASC threat matrix to others within the organization. The recipient has full access to all linked data for this JASC only

Heli. Stat™ Share Message The sender can state why s/he is sending this report, ask a specific question or request a general comment.

Heli. Stat™ Maintains Out-Mail List and Tracks Recipients’ Opening Dates ALERT keeps track for the sender: • Recipient • ATA/JASC code • Model • Date sent Unopened link • Date opened

Heli. Stat™ Enables Early Decision Making 1. For the model and each ATA/JASC component Heli. Stat™ gives the user : • Priority traffic light • Risk assessment matrix • Time trends • Historically unusual occurrences • Source narratives 2. Users have more time to target their activity and to explore source data

Heli. Stat™ Analyses Give More Time for SMEs to Use Expertise

Possible Reactions to Heli. Stat™ Information • • Problem already known and taken care of Check fleet now/soon Consider preventive suggestion Modify procedure: installation/inspection Modify inspection: method or frequency Recommend as audit topic Place on MEL Await further information, tickle

Heli. Stat™- Features • • • Point and click – no programming required Push technology- subscriber specifies model, frequency of reports Mechanical problems affecting safety in 2 data bases(SDR, MMIR) Near real time- reports daily, weekly, monthly E-mail for newly reported events, Risk matrices for ATA/JASC system – 488 codes Trajectory: likelihood and severity over 3 time periods Likelihood rates compared to system-wide benchmarks Likelihood and hazards quantified Past history since 1/1/1996 Narratives of reports for past 182 days

Heli. Stat™ Helps a Safety Culture Evolve Faster SMS becomes more effective when Heli. Stat™ overcomes SMS challenges for : • Becoming proactive • Risk assessment • Prioritization Making SMS more effective helps move the organization towards achieving a safety culture

Heli. Stat™ References 1. Rotor magazine (Winter 2004 -2005, 42 -43 2. Air Beat (January-February 2005), 42 -43 3. Aviation Maintenance, December 2004, 34 4. Air Safety Week, September 27, 2004 5. www. helistat. com

Questions ? Algo. Plus Consulting Limited Suite 308 5670 Spring Garden Road Halifax, NS, Canada B 3 J 1 H 6 http: //www. helistat. com telephone: 1 -902 -423 -5155 1 -888 -ALGOPLUS fax: 1 -902 -484 -7061 email: arichman@algoplusaviation. com