Expert Operator Decision Making Introduction EMS User Conference

Expert Operator Decision Making Introduction EMS User Conference Prepared By Robin Podmore, Inc. Sys Robert Eubank, WECC Greg Lange, GCPUD Frank Greitzer, PNNL

Outline • • State of Applications What is an expert? Knowledge Capture and Transfer Engineer Operator Gap Expert Operator Decision Model (ODM) Simulator Configurations Conclusions

Power System States NORMAL INSECURE Preventative Action Outage, Load Increase Remedial Action Island Blackout RESTORATIVE Restoration Action EMERGENCY

State of Applications • Normal and Insecure State – – – State Estimator Operator Power Flow Contingency Analysis – limited deployment Security Constrained Economic Dispatch Automatic Voltage Control • Emergency State – Load shedding – What Other? • Restoration State – What?

Objectives • Report on past and planned advances in operator training since Blackout of 2003. • Focus on understanding how an expert operator makes decisions. • How can this understanding be applied to development of improved user interfaces and applications for insecure, emergency and restoration states?

Presenters • • Greg Lange, Grant County PUD Robert Eubank – Training Instructor, WECC Dr. Frank Greitzer – Scientist, PNNL Robin Podmore - Inc. Sys

Experts What is an “Expert”? “Someone widely recognized as a reliable source of technique or skill whose faculty for judging or deciding rightly, justly, or wisely is accorded authority and status by the public or their peers. An expert, more generally is a person with extensive knowledge or ability in a particular area of study. ” Source: Wikipedia, 2008

Levels of Expertise Dreyfus and Dreyfus 1986 • Stage 1 Novice – Explicit rules susceptible to context and nuance • Stage 2 Advanced Beginner – Nuance and context begin to be recognized and incorporated • Stage 3 Competence - Transition from calculated effort to intuitive solutions • Stage 4 Proficiency – Scenarios are now being recognized as whole parts, some analysis and conscious choice remains • Stage 5 Expertise – Complete contexts are recognized and performance is fluid and unselfconscious. • Dreyfus, H. L. , & Dreyfus, S. E. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. New York: Free Press.

Novices vs Experts Novice Pattern Expert Pattern Read Analyze Explore Plan Implement Verify

Explicit and Tacit Knowledge • The knowledge in your operating department can be categorized as Explicit knowledge and Tacit knowledge. • Explicit knowledge is formal and systematic. It can be easily communicated and shared, in operating procedures, disturbance reports, databases or a computer program. • Tacit knowledge is highly personal. It is hard to formalize and difficult to communicate to others. • “We know more than we can tell”. • Tacit knowledge is deeply rooted in action, experience, and operator’s commitment to his profession and craft.

Tacit to Tacit Knowledge Transfer • Sometimes one individual shares tacit knowledge directly with another. • For example, when an operator in training (OIT) works side by side with an experienced operator through on the job training. • He learns his tacit skills through observation, imitation and practice. But on its own, socialization or tribal learning is a rather limited form of knowledge creation. • The apprentice learns the master’s skills. But neither the apprentice nor the master gain any systematic insight into their craft knowledge. • Because their knowledge never becomes explicit, it cannot easily be leveraged by the organization as a whole.

Explicit to Explicit Knowledge Transfer • An individual can also combine or apply discrete pieces of explicit knowledge into a new whole. For example, an operator may apply some well documented switching principles to develop a specific switching order. • Compilation all the key papers on Restoration into the IEEE “Power System Restoration” Book. • Development of Scenarios to illustrate operating principles in NERC Manual

From Tacit to Explicit • The development of Explicit knowledge from Tacit knowledge is often one of the more difficult transformations. • With Simulation the process becomes systematic. Scenarios provide a context for the master to express his knowledge. They stimulate the master’s memory on past operating incidents. • When the master operator is presented with a challenging operating condition on the Simulator; he uses his tacit knowledge and experience to develop a specific set of operating actions. • Actions of the master operator and how he responds to solve various problems can be recorded • When the operating principles that the master operator was intuitively applying in developing and selecting these operating actions, can be documented and explained, this knowledge can be shared with other operating staff.

From Explicit to Tacit • As new explicit knowledge is shared throughout the operating department, other operating staff begin to internalize it – that is, they use it to broaden, extend and reframe their own tacit knowledge. • They adapt the procedures and principles that they have learned and observed under one set of operating conditions and apply them to other operating conditions. • These operators eventually come to take it for granted as part of the background of tools and skills needed to do their jobs.

Building a Spiral of Knowledge • Articulation (converting tacit knowledge to explicit knowledge) and • Internalization (using that explicit knowledge to extend one’s own tacit knowledge base) • These are the critical steps in building a spiral of knowledge. • Both require the active commitment of the individual. • The master has to stretch and grow to communicate more of what he knows. • The apprentice has to stretch and grow to apply the explicit knowledge and extend his own operating skills.

Source: SOS International

The Engineer – Operator Gap • “Stuff You Don’t Learn in Engineering School – Skills for Success in the Real World, by Carl Selinger, IEEE Press, 2004. ” • “Book smart and dumb to the world” • “I can train an operator to be an engineer, I can’t train an engineer to be an operator. ” • “Four year graduate engineers are not recruited for system operators. ” • “The rigorous technical Engineering curricula does not allow time for us to teach soft skills. ” • “New graduate engineers will not work shifts. ” • “Applications designed by engineers for engineers. ” • “Our EMS simulator is an engineers dream but an operators nightmare. ”

Engineer – Operator Gap • “Engineering Schools do an inadequate job at producing well-rounded engineers with background courses in English, writing, economics, history and so on. ” • “I have fifty engineers and fifty operators. The engineers, cannot make a real-time decision. ” Chief System Operator of a major RTO. • “MIT is coming to ask us on how they can train there engineers to be leaders. Industry is telling them that there graduates cannot handle operations. ” Dean, United States Naval Academy. • “The IEEE Working Group Report is not readable by operators”

Critical Skills Soft Skills § Broad business understanding § Financial understanding § Interpersonal skills § Strategic thinking and planning § Development § Communications § Decision making § Curiosity § Emotional IQ § Learning Agility § Calm under fire Source: Wanda Reder, IEEE – Experience / Technical Skills § § § § § Power Dispatcher System Operator Relay Technician Underground Technician Transmission Expertise System Planning Engineering Design Engineer Electric Service Line Workers Gas Pressure Control Gas customer Field Service

Important Traits Source J. Theotonio NERC

Crew Resource Management Foundation for Soft Skills • • Situational Awareness Assertiveness Decision Making Communication Leadership Adaptability/Flexibility Mission Analysis

How the Military Bridges the Gap between Officers and Senior Enlisted • Military Academies: One Senior Enlisted per Company. • Prior enlisted can become officers. • Platoon Lieutenants and Sergeants work closely with mutual respect.

Naturalistic Decision Making • Typical industry training doesn’t match the way people think. Effective training for critical decision making must be informed by cognitive theories and models. • Naturalistic Decision Making (NDM) is an area of cognitive science that describes how experienced individuals and teams perform in real-time, mission-critical environments. • First NDM conference held in 1989. • Ninth NDM conference Covent Gardens June 1990

Example Applications of NDM • • • Health Care – Surgeons, Nurses Command Control Aviation Business and Industrial Applications Process Control Material Requirements Planning Nuclear Power Plant Emergencies Anesthesiology Naval Officers in Littoral Environments Skilled Fighter Pilots

Critical Decision Making The Challenge of Decision Making Vague Goals Time Stress Multiple Players High Task Loading Organizational Factors Dynamic Settings Uncertainty High Stakes

NDM Thought Leaders • Gary Klein – Recognition Primed Decision Model • Mica Endsley – Situation Awareness • Marvin Cohen – Consistent, complete, validated “Story” with Quick Test • Sallie Gordon – Cognitive Task Analysis

Recognition Primed Decision Model Developed by Gary Klein Situation That affect the Which you Assess by Mental Simulation Action Script Cues Using your That let you create Mental Models That activate Generates Patterns

NDM Applications to Power • RPDM introduced to Power Industry by Doug Harrington of Team Formation • RPDM extended by Robin Podmore of Inc. Sys, Frank Greitzer of PNNL and Chuck Johansen and Pam Eye of SOS to reflect: – Need for 100% Error Free Decisions – Quick Test – Distinction between Long and Short Term Memory – Endsley’s Three Levels of Situation Awareness

Operator Decision Model Situation Generates That affect the Which you Assess by Mental Simulation Action Script Cues Using your That let you create Mental Models That activate N Time Available? Story (Patterns) Y Which you Validate by Real World Long term memory Mental Simulation Using your Mental Models Short term memory ‹ 34›

‹ 35›

Simulator Configuration Options Control Center Model Generic SCADA/AGC Power System Model Controls Generic Power System Model Generic Simulator Status, Analogs Emulated SCADA/AGC Replicated EMS Controls Status, Analogs Specific Power System Model Custom Simulator Replica EMS Simulator ‹ 36›

CHSAN UXBRDG COPLEY COPMANOR AMUS STNTON OAKDLE 80 200 BEAVER ASH POOL AIRPRT NESTLE TANTON RICTER LOCHER 1200 MOSES 600 ELLS DOYLE CRWFRD 2*400 70 BAKER FARLIE 1200 DAWSON KINCAID WYNHAM 230 KV 138 KV 69 KV 33 KV 400 EXTERNAL GRANGE 600 VEXLEY HOMER 2008 © Copyright Incremental Systems Corporation JENKIN 3*200 500 ‹ 37›

PALCO Map ‹ 38›

Generic Simulator Advantages • Wide Variety of Scenarios, developed and shared by industry leading instructors • Students can learn to operate PALCO in a single training session • Low cost method to obtain NERC mandated: “five days per year of training and drills using realistic simulations of system emergencies” • Catalyst for capturing knowledge of experienced operators. • Clearing house for simulating incidents on real systems • Used by more than 50% certified operators ‹ 39›

PJM: Four Simulation Sessions - Four teams per session – Four operators per team – 36 logged on Users ‹ 40›

Generic Simulators can be used to recruit new engineers and operators ‹ 41›

PG&E CETAC Drills (2009) • 5 weeks • 500 total operators • All major utilities in California • Generic PSM for principles • Custom PG&E Model and Northern California Area for Daylong drills • Drill based on report of earthquake off the coast of N. California 2009 Copyright Incremental Systems Corporation ‹ 42›

‹ 43›

General Process • Industry Standard Format: PSSE or CIM XML • Rapid Building Process • One-line diagrams automatically built • Retains important aspects of external modeling, while reducing the overall model size • Recognizable Station names • Electrically Sound model • Great for verifying blackstart plan • Easily customizable for system operator training PSSE define “Internal” Area Retain All Data “External” Area Retain All Data 200 KV and Above Equivalence And Move Below 200 k. V Auto. Builder Power. Simulator CIM Model ‹ 44›

Station Built Automatically from PSSE Data • Single Bus • Single Breaker • Lower Voltage (under 200 k. V) Equivalenced in the external area • EOP-005 Compliance ready ‹ 45›

Station Customized from PSSE Data • Familiar layouts • Custom Names • Familiar Switching • Transfer Busses • Suitable for: • PER-002 • PER-005 • Blackstart Drills ‹ 46›

External Station Built Automatically from PSSE Data ‹ 47›

Effect of External WECC Model Reduction Power. Simulator performance with the reduced model is very good ‹ 48›

Internal Areas For WECC Models ‹ 49›

Custom Simulator Advantages • Custom models can be developed as an application service. • Dedicated support engineer is not required • Model is customized to represent operator’s own power system. • Complies with PER 005 requirement – “Each RC, BA, TO …. shall provide each System Operator with emergency operations training using simulation technology …. that replicates the operational behavior of the BES …” • Clearing house for simulating incidents on real systems. • Need to OJT to cover differences in Custom Simulator user interface and EMS user interface ‹ 50›

Alberta Electric System Operator Restoration Drills http: //www. powersimulator. net/? m=psm. CENisku. html 2009 Copyright Incremental Systems Corporation ‹ 51›

Types of Knowledge and Skills • Knowledge of power system equipment and principles • Knowledge of Operating Procedures – NERC, Region, Utility • Technical skills at operating A power system • Technical skills at operating YOUR power system • Soft skills: team work, communications, multi-tasking, managing stress and uncertainty • Knowledge of departments/personnel you need to work with. • Knowledge and Skills for operating your EMS and related communication equipment Suited for self study and Virtual Instructor ‹ 52›

Ready with PSD for a Day at School (We were totally safe and protected all the time) ‹ 53›

Iraq Ministry of Electricity National Dispatch Center • Located Al Ameen, Iraq • New and Incumbent Engineers • Generic PSM for principles • Custom Iraq Model • Remedial Action Schemes 2009 Copyright Incremental Systems Corporation ‹ 54›

New Processes for Operator Training • NERC PER 002: – Requirement for 5 days of emergency operations training with simulators • NERC PER 005: – Requirement for Job Task Analysis – Requirement for emergency operations training using • • simulation technology …. that replicates the operational behavior of the BES …” Wide acceptance of Generic Simulators Increasing acceptance of Custom Simulators A Model for Expert Operator Decision Making Cognitive Task Analysis for System Operators ‹ 55›

Can new methods by applied to System Development? • • • Job Task Analysis Generic Simulators Custom Simulators Model for Expert Operator Decision Making Cognitive Task Analysis? • Improved Tools for Normal and In. Secure State • New Tools for Emergency State • New Tools for Restoration State ‹ 56›