Use of Simulation-based Surgical Education and Training within

Use of Simulation-based Surgical Education and Training within the Context of the Core Competencies, Milestones, Patient Safety, and the New ACGME Accreditation System: GENERAL SURGERY Daniel J. Scott, MD, FACS Frank H. Kidd, Jr. , MD, Distinguished Professorship in Surgery Vice Chairman and Surgery Residency Program Director, Southwestern Center for Minimally Invasive Surgery University of Texas Southwestern Medical Center at Dallas

Why is Skill Acquisition an Issue? • Patient safety – Institute of Medicine: To Err Is Human. National Academy of Science, Sept. 1999 • 98, 000 deaths per year due to medical error • “Error in the performance of an operation” cited as one of many causes • New procedures – Advanced laparoscopy – Endovascular – Endolumenal • “Orphan” procedures – e. g. open CBDE, PUD

Why is Skill Acquisition an Issue? • Cost – $53 million annually to train residents in the OR 1 • OR time limitations • Inadequate case volumes • 80 -hour work week constraints • 16 -hour R 1 duty hour limit 1 Bridges, et al. Am J Surg 1999; 177: 28 -32.

UT Southwestern 1998

• Randomized, blinded, controlled trial • 5 hours of videotrainer practice using “Southwestern Stations” • Performance measured during laparoscopic cholecystectomy Training improves operating room performance J Am Coll Surg 2000; 191: 272 -283.

Skills Training: Gaining Momentum 2004 2006 2007 2008 ACS/SAGES FLS Program Validated with high-stakes certification ACS Accredited Education Institutes Standards for Centers of Excellence ACS/APDS National Skills Curriculum High quality simulation exercises with free access RRC Mandate Mandatory skills labs in all residency programs ABS FLS Requirement 2010 FLS Certification required to sit for qualifying examination Surgery RRC

Perfect Storm

Immersion

Immersion Autonomy

Immersion Autonomy Teaching

Immersion Autonomy COMPETENCY Teaching

Are Graduating Residents Competent? • 2012 American Board of Surgery (ABS) Pass Rates – Written Boards: 81% www. abs. org

Are Graduating Residents Competent? • 2012 American Board of Surgery (ABS) Pass Rates – Written Boards: 81% – 19% of graduates have knowledge deficiencies www. abs. org

Are Graduating Residents Competent? • 2012 American Board of Surgery (ABS) Pass Rates – Written Boards: 81% – 19% of graduates have knowledge deficiencies – Oral Boards: 72% www. abs. org

Are Graduating Residents Competent? • 2012 American Board of Surgery (ABS) Pass Rates – Written Boards: 81% – 19% of graduates have knowledge deficiencies – Oral Boards: 72% – 28% of graduates have judgment deficiencies www. abs. org

Are Graduating Residents Competent? • 2012 American Board of Surgery (ABS) Pass Rates – Written Boards: 81% – 19% of graduates have knowledge deficiencies – Oral Boards: 72% – 28% of graduates have judgment deficiencies 28% OF GRADUATES ARE NOT COMPETENT www. abs. org

Are Graduating Residents Competent? • >80% of 1000 graduating residents enroll in fellowships – Lifestyle – Financial reward – Job opportunities in urban areas – Narrow field of expertise is appealing

Are Graduating Residents Competent? • >80% of 1000 graduating residents enroll in fellowships – Lifestyle – Financial reward – Job opportunities in urban areas – Narrow field of expertise is appealing – Not comfortable with their level of competency

Are Graduating Residents Competent? • >80% of 1000 graduating residents enroll in fellowships – Lifestyle – Financial reward – Job opportunities in urban areas – Narrow field of expertise is appealing – Not comfortable with their level of competency – DO THEY NEED MORE TRAINING?

Start Training during Medical School • “The American Board of Surgery recommends and endorses that all incoming surgical residents beginning with the 2014 year complete a preparatory course PRIOR to beginning Surgical Training. ”

More General Surgery Training after Residency • ACS Transition to Practice Program – 6 Programs starting in 2013 – “Help with your transition to independent practice in general surgery” – “Increase your competence and confidence in clinical practice” www. facs. org

UT Southwestern Results 5 -year ABS Pass Rates - First Attempt Both Components UT Southwestern 60 95% xxxxxxxxxxxxxxxx 29 86% xxxxxxxx 35 66% xxxxxxxx 31 84% xxxxxxxx 29 93% xxxxxxxx 27 89% xxxxxxxx 38 84% xxxxxxxx 28 86% xxxxxxxx 44 84% xxxxxxxx 34 85% xxxxxxxx 33 94% xxxxxxxx 25 88% xxxxxxxx 35 74% xxxxxxxx 42 86% xxxxxxxx 51 78% xxxxxxxx 32 81% xxxxxxxx 34 91% xxxxxxxx 41 85% xxxxxxxxxxxxxxxx 25 88% xxxxxxxx 25 84% www. absurgery. org/xfer/fyp 2012 summary. pdf

General Surgery Residency Program Balance Immersive Experience Structured Education

Technical Skills Curricula • General: – – • – – Surgical Airway Basic Ventilator Management Advanced Ventilator Management Ultrasound for Central Line Placement Ultrasound in the ICU Ultrasound for Trauma (FAST) Hemodynamic Monitoring ATOM: Advanced Trauma Operative Management Tracheostomy & PEG Tube Insertion – – – – Percutaneous Breast Biopsy Ultrasound Guided Breast Biopsy Mammosite Module Vascular: – – – Vascular Patch Angioplasty Vascular Anastomosis Vascular Access Laparoscopic/Endoscopic: – – Breast: – – – • Basic Open Knot-tying and Suturing Intermediate Open Knot-tying and Suturing Trauma/Critical care: – – – – • • – – • Lap I Basic Laparoscopic Skills Lap II Basic & Advanced Laparoscopic Skills (FLS) Fundamentals of Laparoscopic Surgery (FLS) Ongoing Training Laparoscopic Common Bile Duct Exploration (LCBDE) Laparoscopic Cholecystectomy & Appendectomy Laparoscopic Port Insertion Laparoscopic Energy Sources Laparoscopic Inguinal Hernia Virtual Reality Flexible Endoscopy Surgeon’s Training Endoscopic Proficiency (STEP) SAGES Top 21 SAGES Biliary Grand Rounds Robotic Training: – Comprehensive Console Surgeon Basic Training

Optimizing Skill Acquisition • Deliberate practice – Separated from the real environment (OR) – Specifically designed plan for training – Goal oriented motivation for learning • Structured training – Protected time allocation – Mandatory participation – Performance and attendance tracking • Distributed training – Maximum 1 -1. 5 hours per session – Distributed over days/weeks/months – Superior to massed practice • Alleviates fatigue • Reinforces mapping/integration Ericsson KA. Acad Med 2004; 79: S 70 -S 81. Moulton, et al. Ann Surg 2006; 244: 400 -409. Mackay, et al. Surg Endosc 2002; 16: 957 -961.

Optimizing Skill Acquisition • Proficiency-based training – Avoids the use of arbitrary training endpoints • Such as predetermined duration or number of repetitions • “One size does not fit all” – Tailors the educational experience to the needs of the learner • Practice a little vs. a lot • Optimizes efficiency and time requirements • Maximizes effectiveness uniform end product (skill acquisition) Seymour, et al. Ann Surg 2002; 236: 458 -463. Korndorffer, et al. Arch Surgery 2005; 140: 80 -84. Brunner, et al. J Surg Research 2004; 122: 150 -156. Stefanidis, et al. Surgery 2006; 140: 252 -262.

Open Skills Curriculum: 12 Tasks Palm Needle Driver Knot-tying, No Tension Suturing, Interrupted Knot-tying, Tension Suturing, Running

Open Skills Curriculum • R 1 (n=37) • Proficiency achieved for 100% of the 12 tasks – 12. 7 hours (range 6. 9 -37. 1) – 141 repetitions (range 52 -513) p <0. 001 • Cost: $776 Goova, et al. J Surg Educ 2008; 65: 309 -15

Open Intermediate Skills Curriculum: 6 Tasks Tie at Depth Suture Ligation (Stick tie) Suturing at Depth, Interrupted simple Tie on a Pass Ties in Continuity Atraumatic Tie

Open Intermediate Skills Curriculum • PGY 1 (n=39) • Proficiency reached • 3. 4 ± 3. 8 hours • 29 ± 17 repetitions • Cost • $4 per trainee Mashaud, et al. Surgery (in press)

Proficiency-based Curriculum • PGY 1 (n=37) 2006 -2007 • Cognitive & Skills: Lap I + Lap II (FLS) • FLS – 7. 4 ± 3. 2 hours, 67 ± 28 repetitions • 78% & 100% FLS exam pass rates after Lap I & II P < 0. 001 Peg Transfer Pattern Cut Ligating Loop P < 0. 001 Extracorporeal Passing Score = 270 Intracorporeal Lap II Goova MT, et al. J Surg Research 2008; 144: 224 -225.

Proficiency-based Curriculum Peg Transfer Pattern Cut • PGY 2 -5 (n=54) 2006 -2007 • FLS Skills Ligating Loop – 6. 2 ± 2. 6 hours – 103 ± 28 repetitions Passing Score = 270 • 63% passing at baseline • Extracorporeal 100% FLS exam pass rate Intracorporeal Goova MT, et al, J Am Coll Surg 2008; 207: S 90.

Ongoing FLS Training 2 -Year Analysis Protocol 2006 -2007 • Task 4 & 5 Initial Proficiency-based FLS Training PGY 1 -5 (n=91) • Testing (1 rep) • Re-training if indicated • Ongoing FLS training 2007 -2009 Ongoing Proficiency-based FLS Training PGY 3 -5 (n=33) components over 2 -year period) Mashaud, et al. Surgery 2010; 148: 194 -201. Retention I 6 mo Retention III 6 mo – 96% participation rate (all curricular 6 mo Retention IV PGY 4 -5 Certification (n=20)

Ongoing FLS Training Results Task 4: Performance Improvement and Retention (n=33) PGY 3 83% at 6. 3 months PGY 5 94% at 12. 3 months PGY 4 98% at 18 months 91% at 23. 3 months Mashaud, et al. Surgery 2010; 148: 194 -201.

Ongoing FLS Training Results 550 Task 5: Performance Improvement and Retention (n=33) 500 450 400 350 PGY 3 85% at 6. 3 months 95% at 12. 3 months 300 PGY 5 250 96% at 18 months PGY 4 100% at 23. 3 months 200 Pre-test Post-test Retention 1 Retention 2 Retention 3 Mashaud, et al. Surgery 2010; 148: 194 -201. Retention 4

Ongoing FLS Training Certification Exam PGY 4 -5 (n=20) 92% Retention at 23. 3 months Passing Score Mashaud, et al. Surgery 2010; 148: 194 -201.

Comprehensive Proficiency-based Robotic Training Curriculum • Curriculum Design – 23 unique skills identified through task deconstruction – Residents, fellows, faculty – General Surgery, Gynecology, Urology • Curriculum Components – Online tutorial (MCQ’s, certificate) – ½ day interactive session (global ratings) – 9 inanimate exercises (objective metrics) • Validation (Content, Face, Construct, Reliability) Dulan G, et al. Surgery 2012; 152: 477 -88. Dulan G, et al. Surg Endosc 2012; 26: 1516 -1521. Dulan G, et al. Am J Surg 2012; 203: 535 -539.

Task 1. Peg Transfer Task 2. Clutch/Camera Movement Task 3. Rubber Band Transfer Task 4. Suture (Simple) Task 5. Clutch/Camera Peg Transfer Task 6. Stair Rubber Band Transfer Task 7. Running/Cutting Rubber Band Task 8. Pattern Cut Task 9. Suture (Running)

Performance Improvement (n=53) *p<0. 05 1000 Composite Score 800 600 400 200 0 Pretest Post-test Arain et al. Surg Endosc 2012; 26: 2740 -5.

University of Texas Healthcare Safety and Effectiveness Grant Program Ensuring Competency: Is FLS Certification Necessary for Practicing Surgeons? Melanie L Hafford MD Kent R Van. Sickle MD Ross E Willis Ph. D Todd D Wilson MD Kristine Gugliuzza MD Kimberly M Brown MD Daniel J Scott MD

FLS Certification for Faculty Surgeons • Pretest: n=83 – 26 (31%) failed manual skills • Training P= <0. 001 – Skills: 2. 5 ± 2. 3 hrs. – Cognitive: 2. 6 ± 2. 1 hrs • Certification Examination: n=76 – 1 (1%) failed manual skills – 9 (12%) failed cognitive exam • 6 remediated, passed on re-take Hafford, et al. Surg Endosc 2013; 27: 118 -26.

Conclusions • Numerous threats to developing competency exist • Simulation plays a pivotal role – Technical skills – Judgment (i. e. decision-making)? • Performance benchmarks may facilitate competency assessments in a longitudinal fashion (i. e. milestones)

Thank You!