Dédale EUROCONTROL Systemic Occurrence Analysis Methodology

SOAM A Reason-Based Organisational Methodology A tool for the analysis of safety occurrences (accidents & Incidents)

SOAM Antecedents l The Reason Model ~ circa 1990 è Developed from Professor James Reason’s work on human error and “organisational accidents” l Tripod Delta ~ circa 1994 è Developed Model l for Shell Petroleum, based on Reason ICAM ~ circa 2000 è Developed for BHP Billiton, based on Reason Model and Tripod Delta EUROCONTROL SOAM

“Recent” fatal accidents in Europe with ATM contribution Linate, 8 October 2001 Überlingen, 1 July 2002 EUROCONTROL SOAM

Call for Action l l l Aviation Safety is still at a good level Traffic however is increasing Two major accidents involving Air Traffic Management in 2001 and 2002 …. . In well developed countries in the heart of Europe … From which many lessons can be learnt Wake up call for action EUROCONTROL SOAM

AGAS/ SSAP Priority Areas An Action Group of European aviation safety experts identified the following areas as needing immediate focus: 1. Safety related human resources in ATM 1. Safety related human resources in ATM 2. Incident reporting and data sharing 3. ACAS/TCAS safety nets 4. Ground-based safety nets 5. Runway safety ESARRs and implementation monitoring 6. Enforcement of ESARRs and implementation monitoring 7. Awareness of safety matters 8. Safety and human factors research & development EUROCONTROL SOAM

Why another investigation tool? l l Support ESARR 2 implementation and Strategic Safety Action Plan Higher quality reports and AST returns ~ a need to: è è clearly identify causes and report them concisely go beyond the human errors, to find systemic causes use a simple, consistent approach for events of all severity levels ensure recommendations are relevant and effective EUROCONTROL SOAM

Current investigation methods What can we improve about the way we conduct safety occurrence investigations? EUROCONTROL SOAM

How SOAM can help l A methodology that includes structured processes to: è è è identify and classify a range of contributing factors sort out irrelevant, non-contributing facts move from a focus on human error/s to identify systemic causes ~ support for ‘Just Culture’ analyse simple events through to high severity incidents and accidents clearly link recommendations to the facts of the analysis EUROCONTROL SOAM

Elements of Organisational Occurrences l Organisational Factors: è l Contextual Conditions: è l Situational factors involving characteristics of the task, the environment or human limitations Human Involvement: è l Latent system failures that produce or allow conditions under which accidents are possible Errors and/or violations which have an immediate adverse affect (“active failures”) Inadequate or absent barriers/defences: è Failure to identify and protect the system against human errors or violations, local conditions EUROCONTROL SOAM

The Reason Model Organisational Error Chain Contextual Conditions Organisational and System Factors Human Involvement Absent or Failed Barriers People, Task, Environment e saf “Un ts” Ac ACCIDENT Latent Conditions Active Failures EUROCONTROL SOAM Limited window/s (adapted from of opportunity Reason, 1990)

Accident “Causes” l l l A man has a bad argument with his wife. He storms out of the house to the nearest bar and drinks four whiskies. He then decides to go for a drive. It is night-time, there is a skim of snow on the ground, and the tyres on our victim’s car are smooth. In rounding a poorly banked curve at excessive speed, the right front tyre blows out, the car leaves the road and is demolished. What is the cause of the accident? (Johnston, 1996) EUROCONTROL SOAM

SOAM Worked Example Runway Overrun, Bangkok September 1999 EUROCONTROL SOAM

Accident Summary On 23 September 1999, at about 2247 local time, a Boeing 747 -438 aircraft overran runway 21 Left (21 L) while landing at Bangkok International Airport, Thailand. EUROCONTROL SOAM

Accident Summary The overrun occurred after the aircraft landed long and aquaplaned on a runway which was affected by water following very heavy rain. The aircraft sustained substantial damage during the overrun. None of the three flight crew, 16 cabin crew or 391 passengers reported any serious injuries. EUROCONTROL SOAM

SOAM analysis key steps Review the Facts Identify the Absent or Failed Barriers Identify the Human Involvement Identify the Contextual Conditions Identify the Organisational Factors Validate the OFs against the Occurrence “CHECK QUESTIONS” HELP TO SORT AND CLASSIFY FACTS

HARDWARE Equipment, vehicles, tools, controls, switches, levers, workplace design, seating etc S H E SOFTWARE LIVEWARE ENVIRONMENT Procedures, checklists, manuals, training materials, charts etc Operators Site, terrain, weather, roads, traffic, remoteness etc L LL LIVEWARE/LIVEWARE Interface between people. Operators, controllers, managers, etc EUROCONTROL SOAM The SHEL Model (after Edwards, 1972)

Raw Data Collection AC 1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway. PEOPLE HARDWARE SOFTWARE ENVIRONMENT ORGANISATION Crew employed flaps 25/ idle. Captain landing reverse configuration Normal practice to use flaps 25/idle reverse Very heavy rainfall, runway surface affected by water Introduction of new landing procedure poor FO did Officer the First not fly aircraft accurately during final approach Importance of reverse thrust as stopping force on water-affected runways not known Revised approach/ landing procedure introduced in 1996: flaps 25, idle reverse thrust Reduced visibility & distraction: rain and windscreen wipers No formal risk assessment conducted when changed landing procedure researched Captain cancelled go. Other pilots around decision by retarding thrust levers FO awake for 19 hours at the time of the accident Captain did not order a go-around earlier Recent crew experience using full reverse thrust lacking Crew did not use an adequate risk mgt strategy for approach and landing Captain awake 21 hours at time of accident Captain & FO quite low levels of flying prior 30 days Most pilots not fully aware about 'aquaplaning' Confusion after thrust levers retarded, in high workload situation Boeing advised that if idle reverse technique is adopted, it should be the exception rather than the rule Absence of reverse thrust during landing roll noticed, not used No appropriately documented info, procedures regarding operations on wateraffected runways No policies, procedures on duty or work limits for pilots with flying & nonflying duties Qantas B 747 s generally operated in good weather & to aerodromes with long, good quality runways Bangkok runway was resurfaced in 1991 Documents unclear (eg. , key terms not well defined) High workload situation, distraction or inexperience Most pilots disagreed they had adequate training on landing on contaminated runways Partial loss of external visual reference due to heavy rain No policies or procedures for maintenance of recency for management pilots Cost-benefit analysis of new landing procedure was biased Contaminated runway issues not covered in recent years during crew endorsement, promotional or recurrent training “Landing on Slippery Runways” (Boeing doc) not distributed in Qantas since 1977 No formal review of new procedures after 'trial' period Gather data relevant to the occurrence

Raw Data Refinement AC 1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway. PEOPLE HARDWARE SOFTWARE ENVIRONMENT ORGANISATION Crew employed flaps 25/ idle. Captain landing reverse configuration Normal practice to use flaps 25/idle reverse Very heavy rainfall, runway surface affected by water Introduction of new landing procedure poor FO did Officer the First not fly aircraft accurately during final approach Importance of reverse thrust as stopping force on water-affected runways not known Revised approach/ landing procedure introduced in 1996: flaps 25, idle reverse thrust Reduced visibility & distraction: rain and windscreen wipers No formal risk assessment conducted when changed landing procedure researched Captain cancelled go. Other pilots around decision by retarding thrust levers FO awake for 19 hours at the time of the accident Captain did not order a go-around earlier Recent crew experience using full reverse thrust lacking Crew did not use an adequate risk mgt strategy for approach and landing Captain awake 21 hours at time of accident Captain & FO quite low levels of flying prior 30 days Most pilots not fully aware about 'aquaplaning' Confusion after thrust levers retarded, in high workload situation Boeing advised that if idle reverse technique is adopted, it should be the exception rather than the rule Absence of reverse thrust during landing roll noticed, not used No appropriately documented info, procedures regarding operations on wateraffected runways No policies, procedures on duty or work limits for pilots with flying & nonflying duties Qantas B 747 s generally operated in good weather & to aerodromes with long, good quality runways Bangkok runway was resurfaced in 1991 Documents unclear (eg. , key terms not well defined) High workload situation, distraction or inexperience Most pilots disagreed they had adequate training on landing on contaminated runways Partial loss of external visual reference due to heavy rain No policies or procedures for maintenance of recency for management pilots Cost-benefit analysis of new landing procedure was biased Contaminated runway issues not covered in recent years during crew endorsement, promotional or recurrent training “Landing on Slippery Runways” (Boeing doc) not distributed in Qantas since 1977 No formal review of new procedures after 'trial' period Sort out the non-contributing facts of the investigation

Raw Data Refinement AC 1 overruns runway at Bangkok after landing long, recent heavy rainfall, and water on runway. PEOPLE HARDWARE SOFTWARE ENVIRONMENT ORGANISATION Crew employed flaps 25/ idle. Captain landing reverse configuration Normal practice to use flaps 25/idle reverse Very heavy rainfall, runway surface affected by water Introduction of new landing procedure poor FO did Officer the First not fly aircraft accurately during final approach Importance of reverse thrust as stopping force on water-affected runways not known Revised approach/ landing procedure introduced in 1996: flaps 25, idle reverse thrust Reduced visibility & distraction: rain and windscreen wipers No formal risk assessment conducted when changed landing procedure researched Captain cancelled go. Other pilots around decision by retarding thrust levers FO awake for 19 hours at the time of the accident Captain did not order a go-around earlier Most pilots not fully aware about 'aquaplaning' Confusion after thrust levers retarded, in high workload situation No appropriately documented info, procedures regarding operations on wateraffected runways No policies, procedures on duty or work limits for pilots with flying & nonflying duties Qantas B 747 s generally operated in good weather & to aerodromes with long, good quality runways Recent crew experience using full reverse thrust lacking Documents unclear (eg. , key terms not well defined) High workload situation, distraction or inexperience Crew did not use an adequate risk mgt strategy for approach and landing Most pilots disagreed they had adequate training on landing on contaminated runways Partial loss of external visual reference due to heavy rain Captain awake 21 hours at time of accident Captain & FO quite low levels of flying prior 30 days Absence of reverse thrust during landing roll noticed, not used No policies or procedures for maintenance of recency for management pilots Cost-benefit analysis of new landing procedure was biased Contaminated runway issues not covered in recent years during crew endorsement, promotional or recurrent training “Landing on Slippery Runways” (Boeing doc) not distributed in Qantas since 1977 No formal review of new procedures after 'trial' period Use the remaining factors to build the Analysis chart

Building the Analysis Chart ORGANISATIONAL FACTORS CONTEXTUAL CONDITIONS HUMAN INVOLVEMENT ABSENT OR FAILED BARRIERS Very heavy rainfall, runway surface affected by water ? ACCIDENT ? Very heavy rainfall, runway surface affected by water EUROCONTROL SOAM

Absent or Failed Barriers l l Describe the “last minute” measures which failed or were missing, and therefore did not prevent the accident Check Question: “Does the item describe a work procedure, aspect of human awareness, physical obstacle, warning or control system, or protection measure designed to prevent an occurrence or lessen its consequences? ” EUROCONTROL SOAM

Human Involvement l l Describe the errors or violations (actions or omissions) by operators at the scene which “triggered” the accident Check Question: “Does the item describe an action or nonaction (error or violation) that immediately contributed to the occurrence? ” EUROCONTROL SOAM

Building the Analysis Chart ORGANISATIONAL FACTORS CONTEXTUAL CONDITIONS Very heavy rainfall, runway surface affected by water HUMAN INVOLVEMENT ABSENT OR FAILED BARRIERS Crew employed flaps 25/ idle reverse landing configuration ? Crew employed flaps 25/ idle reverse landing configuration EUROCONTROL SOAM ACCIDENT

Contextual Conditions l l Describe the context of the event ~ the conditions existing immediately prior to, or at the time of the accident Check Question: “Does the item describe an aspect of the workplace, local organisational climate, or a person’s attitudes, personality, performance limitations, physiological or emotional state that helps explain their actions? ” EUROCONTROL SOAM

Organisational Factors l l Describe the organisational and system factors (failures) which created, or allowed, the prevailing contextual conditions Check Question: “Does the item describe an aspect of an organisation’s culture, systems, processes or decision -making that existed before the occurrence and which resulted in the contextual conditions or allowed those conditions to continue? ” EUROCONTROL SOAM

OTHER SYSTEM FACTORS AC CASA surveillance of airline flight operations deficient CONTEXTUAL CONDITIONS PP No appropriately documented info, procedures re operations on water-affected runways PP Regulations covering contaminated runway operations deficient ORGANISATIONAL FACTORS Very heavy rainfall, runway surface affected by water CO “Landing on Slippery Runways” (Boeing doc) not distributed in Qantas since 1977 TR Contaminated runway issues not covered during crew endorsement, promotional or recurrent training in recent years CO Documents unclear (eg. , key terms not well defined) Crew not aware of critical importance of reverse thrust as stopping force on water-affected runways Most pilots not fully aware about 'aquaplaning' New 1996 approach/ landing procedure inappropriate RM No formal risk assessment conducted when changed landing procedure researched PP Regulations covering emergency procedures & EP training were deficient Normal practice to use flaps 25/idle reverse CM No formal review of new procedures after 'trial' period Recent crew experience using full reverse thrust lacking Reduced visibility & distraction: rain and windscreen wipers CG Cost-benefit analysis of new landing procedure was biased Captain & FO quite low levels of flying prior 30 days WM No policies or procedures for maintenance of recency for management pilots FO awake for 19 hours at the time of the accident WM No policies, procedures on duty or work limits for pilots with flying & non-flying duties Flight crew did not use an adequate risk management strategy for approach and landing Qantas B 747 s generally operated in good weather & to aerodromes with long, good quality runways OC Mgt decisions informal, “intuitive”, “personality-driven” CM Introduction of new landing procedure poor HUMAN INVOLVEMENT Captain awake 21 hours at time of accident High workload situation EUROCONTROL SOAM Crew employed flaps 25/idle reverse landing configuration First Officer did not fly the aircraft accurately during the final approach Captain did not order a goaround earlier Captain cancelled go-around decision by retarding the thrust levers ABSENT OR FAILED BARRIERS ACCIDENT SOAM Chart Aircraft Accident Boeing 747 -438 Bangkok, Thailand September 1999 Landing procedure inappropriate Absence of reverse thrust during landing roll noticed, reverse thrust not used Crew Resource Management deficient • Aircraft overran runway after landing long • No serious injuries (391 pax, 19 crew) • Potential for more serious outcome • Aircraft repair cost: $100, 000 (? ) • Damage to company reputation

Recommendations l l Provide recommendations that will prevent recurrence of this scenario Recommendations should be directed to the responsible position, and must address all identified: 1 Absent or Failed Barriers 2 Organisational Factors EUROCONTROL SOAM

OTHER SYSTEM FACTORS AC CASA surveillance of airline flight operations deficient CONTEXTUAL CONDITIONS PP No appropriately documented info, procedures re operations on water-affected runways PP Regulations covering contaminated runway operations deficient ORGANISATIONAL FACTORS Very heavy rainfall, runway surface affected by water CO “Landing on Slippery Runways” (Boeing doc) not distributed in Qantas since 1977 TR Contaminated runway issues not covered during crew endorsement, promotional or recurrent training in recent years CO Documents unclear (eg. , key terms not well defined) Crew not aware of critical importance of reverse thrust as stopping force on water-affected runways Most pilots not fully aware about 'aquaplaning' New 1996 approach/ landing procedure inappropriate RM No formal risk assessment conducted when changed landing procedure researched PP Regulations covering emergency procedures & EP training were deficient Normal practice to use flaps 25/idle reverse CM No formal review of new procedures after 'trial' period Recent crew experience using full reverse thrust lacking Reduced visibility & distraction: rain and windscreen wipers CG Cost-benefit analysis of new landing procedure was biased Captain & FO quite low levels of flying prior 30 days WM No policies or procedures for maintenance of recency for management pilots FO awake for 19 hours at the time of the accident WM No policies, procedures on duty or work limits for pilots with flying & non-flying duties Flight crew did not use an adequate risk management strategy for approach and landing Qantas B 747 s generally operated in good weather & to aerodromes with long, good quality runways OC Mgt decisions informal, “intuitive”, “personality-driven” CM Introduction of new landing procedure poor HUMAN INVOLVEMENT Captain awake 21 hours at time of accident High workload situation EUROCONTROL SOAM Crew employed flaps 25/idle reverse landing configuration First Officer did not fly the aircraft accurately during the final approach Captain did not order a goaround earlier Captain cancelled go-around decision by retarding the thrust levers ABSENT OR FAILED BARRIERS ACCIDENT SOAM Chart Aircraft Accident Boeing 747 -438 Bangkok, Thailand September 1999 Landing procedure inappropriate Absence of reverse thrust during landing roll noticed, reverse thrust not used Crew Resource Management deficient • Aircraft overran runway after landing long • No serious injuries (391 pax, 19 crew) • Potential for more serious outcome • Aircraft repair cost: $100, 000 (? ) • Damage to company reputation

Questions? EUROCONTROL SOAM

EUROCONTROL SOAM