Training Situation Assessment and Decision Making A rule

Training Situation Assessment and Decision Making A rule making approach Rene Amalberti, MD, Ph. D Professor of Medicine, Physiology and ergonomics, Head Cognitive Science Department, IMASSA, Test-flight center, Brétigny-sur-Orge Ramalberti@imassa. fr Amalberti 1

Outline of the presentation n n Basic Assumptions on Training and Decision Making in Aviation The Reality gap Tactical and strategic options for rule making The JAA approach Amalberti 2

Basic Assumptions n n Situation assessment and decision making are priorities for flight safety (CAST, JSSI, ALAR) Specific training is needed Amalberti 3

Situation Assessment and Decision Making are priorities for Safety (CAST, JSSI, ALAR Fatalities by Accident Categories Fatal Accidents, Worldwide Commercial Jet Fleet (1990 -1999) Approach & Landing Accident Reduction Amalberti 4

Approach & Landing Accident Reduction ATC n Challenge or refuse ATC instructions when they are not clearly understood, are questionable or conflict with your assessment of aircraft position relative to the terrain. n Exercise good radio communication discipline. n Know the height of the highest terrain or obstacle in the operating area. n Know your aircraft’s position in relation to the surrounding high terrain. Amalberti 5

Approach & Landing Accident Reduction Autoflight System n n Monitor the autoflight system for desired operation. Use the best available mode for current flight conditions. Follow procedures. Monitor navigation performance. Amalberti 6

CRM, SOPs and Training Factors Involved in ALAs Approach & Landing Accident Reduction q 74% - Inadequate crew decision making q 72% - Inadvertent non adherence to procedures q 63% - Failure in CRM (cross-check/coordination) q 46% - Failures in company management q 40% - Deliberate non adherence to procedures q 37% - Inadequate training CRM = Crew resource management SOPs = Standard operating procedures Amalberti 7

The reality gap n n n Training time costs money, and tends to remain as short as possible Consistent plan for integrated human factors training, conceptually and practically hard to design n Persistent false believes on cognition, n human behaviour, and human error Training evaluation for human factors related aspects hard to accept Amalberti 8

Reduction time in training courses Feedback from the FAA/JAA report on The Interfaces Between Flight crews and Modern Flight Deck Systems, June 1996 n Reduction time n n n Amalberti Strong belief (evidence? ) that learning cognitive skills needs a shorter time compared to learning motor skills Back to the minimum Ambiguities for decision making 9

Persistent false belief on Human behaviour and human error n Training priorities not necessary the ones identified – – – False belief regarding « back to manual » Remaining language problems aggravating situation awareness Effects of complexity on pilots’ expertise Amalberti 10

Poor situation awareness and false expectation on crew behaviour • 10 crews experiencing rare software failures • Modification of Flight Simulator Capacities • LOFT scenario: LYON - MADRID Speed corruption Mach corruption Altitude ineffective preset Automatic reversion of descent mode between Heading. Vertical Speed (Hdg. Vs) and Track-Flightpath angle (TK-FPA) and versus. Amalberti 11

Experimental crew training to deal with automation surprises (Results 1) n n n n All crews landed safely Only four crews went back to manual All crews showed a tendency to cycle the failing system several times to test the failure before trying to reset the system in order to recover it (resetting fuse, proceeding the global function reset, e. g. resetting FMGS) A thorough analysis of crews’ mental representation and situation awareness via verbal protocols analysis show a series of unexpected ‘magic’ thinking on sub-systems organisation and inter-relations. Crews gave priority to the situation management, not to optimal error or failure management. HE rate increases to 50%, error detection rate going down to 30%, only 10% of HE properly recovered. No causal reasoning, search for comprehension collectively refrained First priorities for safe situation management : control the very short term, and search for solutions compliant with the goal. Amalberti 12

Complexity, and Expertise Glass-cockpit a/c, Amalberti, 1996, 2000 Virtual space of total system knowledge Knowledge at the end of A/C type-rated Initial phase Routines Exploration phase Cognition in the wild Retraction phase Core of routines Amalberti 13

New Technologies and Associated Cultural Problems APPR PROC: STATUS -L/G……………ON -APPR NAVAID…. ON RMP 1 APPR SPD: VREF +10 KT LDG DIST…………X 1. 55 ENG 1 APPR IDLE ONLY ENG 2 APPR IDLE ONLY ALTN LAW: PROST LOST WHENL/G ON: DIRECT LAW CTR TK FUEL UNUSABLE PACKS AT FIXED TEMP INCREASED FUEL CONSUPP SLATS/FLAPS SLOW CAT 1 ONLY TAT +28°C SAT +16°C 11 H 05 INOP SYS WINDSHEAR DET ATT LIMIT OVSPD LIMIT ADR 2+ RA 1+2 SPLR 1+2+5 ELAC 2 SEC 2+3 ILS 2 FAC 1+2 A/CALL OUT VHF 2 ACP 3 CAPT STAT CAPT TAT WSHLD HEAT GW XX KG Amalberti 14

Performance and technology induced problems Performance More solutions? Comprehension Management Back to manual? Negative feedbacks SMART Nearloss LOSS OF CONTROL NEEDED Negative feedbacks Workload Management Amalberti 15

Tactical and strategic options for rule making ointviation A A uthorities n n Cultural approach to rule making All training should be evaluated, including Human factors training n n ICAO annexes A good training should combine basic knowledge (classroom), and practice. Amalberti 16

ointviation A A uthorities q q Cultural differences for rule making The southern Mediterranean way of thinking q Rules are objectives, goals, stimuli to change behaviours q Rules are promoted when no –or very few- actors can comply with q Recommendations are useless The US and Northern Europe way of thinking q Rules are references q They are set when all of the field actors can comply with q Recommendations prepare field actors, well before the rule, to comply with the future rule Amalberti 17

Competing philosophies for training design Philosophy 1 : The NORMATIVE APPROACH Philosophy 2 : The ECOLOGICAL APPROACH Aviation operations can be entirely specified through standardized procedures, programs, schedules, rules, nominal tasks, certification, selection, norms Aviation operations cannot be entirely specified through standardized procedures, programs, and the like. One reason is it includes Humans. Safety improvement will result from more specification and more discipline from the operators; Safety improvement will result from a better respect of the “ecology” of the system and a better acknowledgment of its self-protection mechanisms Deviations from nominal operation are both a cause of lower performance, and the main threat for safety Deviations from nominal operation are both a necessity for adaptation to random dimension of real life, and a potential threat for safety Human operators are ultimately the only unpredictable and unspecifiable components of the system. They are the main source of deviation; Human operators are up to now the only intelligent, flexible and real time adaptable component of the system. They are a deposit and source of safety Errors are non intentional but regrettable deviations from standard actions. Errors are unfortunately inevitable Errors are deviations from operator’s intentions, but at the same time they are part of the normal process of achieving intentions. Errors are necessary The human operator is one more “black-box” coupled through inputs (perceptive data) which are transformed into outputs (actions) according to specified targets (goals) using adequate transfer functions (procedures, skills, . . ) Human operators are auto-organized structures, coupled through recursive processes of self regulation, and ultimately governed by their internal intentions Amalberti 18

ointviation A A uthorities q q JARs / HF TRAINING JAR FCL Cockpit crew training q 1. 160, 1470 Ab initio : theoretical knowledge in HF / Theoretical HF examination to get the ATPL - Compliant with ICAO annex 1 q 1. 240 Flight check licensing : demonstrating ability in crew co-ordination and airman ship, requirements extended into MCC regulations JAR OPS Recurrent training Cockpit & Cabin Crew q 1. 940, 45, 55, 65 CRM for cockpit crews. Recent change of requirements following the NPA 16, compliant with ICAO annex 6 q CRM for Cabin crew : TGL 5 in progress Amalberti 19

Zoom on HF training flight crew & on the NPA 16 CRM Cockpit crew Recurrent CRM 3 Yrs cycle Modular/LOFT ATPL JAR FCL 1 st year HPL JAR OPS MCC Initial CRM Theoretical course in Human Factors according JAR FCL Amalberti Conversion course CRM training when changing operator Conversion course CRM training when changing aeroplane type Command course CRM training 20

NPA 16. Appendix 1. 965 New need for CRM assessment Amalberti 21

The NOTECHS project “Non-technical skills refer to a pilot’s attitudes and behaviours in the cockpit not directly related to aircraft control, system management, technical consequences and standard operating procedures. ” THE NOTECHS PROJECT 1997 -1998 – – n Sponsored by 4 CAAs [NL, D, UK, &F], and DGVII Basic design and development NLR (J. van Avermaete, E. Kuriijsen, THE JARTEL PROJECT 1999 -2001 – – n n DLR ( K-M Goeters, H-J. Hormann) Aberdeen University (R. Flin, L. Martin) n Same teams n NLR (J. van Avermaete, E. Kuriijsen, H. Nijhuis + KLM Pilots), project leader n Sponsored by DGTREN Validation & benchmarking n n Plus n IMASSA (R. Amalberti & C. Valot) n n n Amalberti DLR ( K-M Goeters, H-J. Hormann) Aberdeen University (R. Flin, L. Martin) IMASSA (R. Amalberti & C. Valot) n British Airways Alitalia Airbus DERA Sofreavia (project leader) 22

NOTECHS Principles for system design and operation FOUR PRINCIPLES FOR DESIGN q Maximum mutual exclusivity for categories and elements q Rule of parsimony - minimalist approach q Everyday terminology q Skills directly observable or inferred from other behaviours (communication) FOUR PRINCIPLES FOR OPERATIONS, designed to ensure that each crewmember receives as fair and as objective an assessment as possible. q Only observable behaviour is to be assessed The evaluation must exclude reference to a crewmember’s personality or emotional attitude and should be based only on observable behaviour. Behavioural markers were designed to support an objective judgement. q Need for technical consequence For NTS to be rated as unacceptable, flight safety must be actually (or potentially|) compromised. Thus demanding a related objective technical consequence. q Repetition required Repetition of unacceptable behaviour during the check must be observed to conclude that there is a significant problem. If, according to the JAR-paragraph concerned, the nature of a technical failure allows for a second attempt, this should be granted, regardless of the NTS rating. q Explanation required For each Category rated as unacceptable the examiner must: Indicate the Element(s) in that Category where the unacceptable behaviour was observed. b. Explain where the observed NTS (potentially) led to safety consequences. Give a free-text explanation on each of the Categories rated unacceptable, using standard phraseology. Amalberti 23

The NOTECHS framework Non-technical skills Co-operation Leadership & management skills Team building & maintaining Conflict solving Decision making Category Element Considering others Supporting others Situation awareness Behaviour Helps other crew members in demanding situations Offers assistance Amalberti 24

Framework: Elements & Behaviours for Category - Situation Awareness Category: Situation Awareness Elements System Awareness Environmental Awareness Anticipation Behaviours - Monitors and reports changes in systems states - Acknowledges entries and changes to systems - Collects information about the environment - Contacts outside resources when necessary - Shares information about the environment with others Amalberti - Discusses contingency strategies - Identifies possible/future problems 25

Conclusion n Significant changes in the past decade Extension to Cabin crew, Maintenance, etc, in progress Remaining problems for success ü ü Ends : Adhere to procedure Vs teach adaptation Cost and fragility : what about HF training courses after the initial burst of enthusiasm ? Means : who should be the instructors ? (good recent UK contribution) Evaluation : Can we design HF training without evaluation? Amalberti 26