Bridge over Troubled Water Linking UME and GME

Bridge over Troubled Water Linking UME and GME Monica L. Lypson, MD Assistant Dean, Graduate Medical Education University of Michigan • Jeff Fabri, MD Rita M. Patel, MD

Education Across the Continuum Standardization of Assessment Student Portfolios to Baseline Assessment Monica L. Lypson, MD Assistant Dean, Graduate Medical Education University of Michigan • • Associate Chief of Staff, VA Ann Arbor Health Care System

Institutional OSCE – Post-Graduate Orientation Assessment (POA) · Developed by the Graduate Medical Education Committee (GMEC) at the University of Michigan · It is our initial step in training our residents · It was established to determine residents’ baseline proficiency in particular aspects of the ACGME’s six general competencies Lypson ML. et. al. Academic Medicine. 79(6): 564 -70, 2004 Jun.

POA CONTENT · Knowledge and skills needed during the first six to eighteen weeks of residency/internship · Emphasizes clinical situations that are often encountered without formal supervision · 9 Assessment & Educational Stations

The POA as Formative Assessment · Results of the POA determine the basis for individualized “learning agendas” · Remediation is provided after the completion of each station · Residents received educational materials that provide the “answers” to the information assessed during the POA · Standardized Patient feedback is provided to the program director within 24 hours if the resident performs exceptionally well or poorly during the POA

Implementation · · · Administered over 4 days of Paid hospital orientation Provided to approximately 150 PGY-1 residents in over 15 specialties Cost Approximately $250 per resident for the assessment - $500 if you add in salary There are some discipline specific scenarios – e. g. Pediatric cases and examples Scores are provided to the resident and program director within 7 -10 days of the POA

Subspecialty Programs Dentistry • Emergency Medicine • Family Medicine • General Surgery • – Preliminary Residents • Internal Medicine – Preliminary Residents • • Otolaryngology Orthopedics Pathology Pediatrics Plastic Surgery Psychiatry Urology PGY-2 s Internal Medicine – • Pediatrics (Residents that did not complete their Internships at UMHS) – Physical Medicine & Rehabilitation • Neurosurgery – Dermatology – Neurology • Obstetrics & Gynecology •

Station Content • Critical Values / Multiple Choice – Computer Based assessment – Review and diagnose 17 short patient scenarios – Multiple Choice Format similar to United States medical Licensing Exam (USMLE) Step 3 – Competencies: • Patient Care • Medical Knowledge • Socio-Cultural Communication – Standardized patient assessment – Assesses the understanding of disease and treatment recommendations in the context of the patient’s health beliefs and socio-economic setting – Competencies: • Interpersonal and Communication Skills • Professionalism • Patient Care • Practice-Based Learning and Improvement

Station Content • Evidence-Based Medicine (EBM) – Computer-based Assessment – Generate a clinical question – Residents reviewed abstracts & identify the appropriate treatment – Competencies: • Practice-based Learning and Improvement • Medical Knowledge • Images -Rays) (X – Computer-based Assessment – Review & diagnose 18 images – Many of the common films reviewed in the middle of the night while “on call” – Competencies: • Patient Care • Medical Knowledge

Station Content • Informed Consent – Standardized patient Assessment – Obtain informed consent from a patient for a procedure – Hospital & JCAHO standards and policies – Competencies: • Interpersonal and Communication Skills • Professionalism • Patient Care • Patient Safety – Pen & Paper Assessment – Order Writing Station • Legibility • Signature, Date, Time – Respiratory Distress • Treatment of the acute Asthmatic – Competencies: • Systems-Based Practice • Professionalism • Patient Care • Medical Knowledge

Station Content • Aseptic Technique – Checklist Evaluation by Expert Nursing Staff and Standardized Patients – Create & maintain a sterile field while performing a “mock” I&D – Universal Protocol & “Time-out” procedures covered – Assessment/Remediation materials include a review of JCAHO requirements – Improved nursing and house officer interactions – Competencies: • Patient Care • Medical Knowledge • Systems-Based Practice

Station Content • System Compliance / Fire Safety Station – Surgical Fire Safety (20/20) – Housestaff involvement with a Patient fire – Questions covering use of safety equipment & review of JCAHO requirements – Video & Computer Based Assessment – Competencies: • Patient Care • Systems-Based Practice • Pain Assessment – Pain Assessment Tools – Educates on the appropriate medications to use for pain – Explains hospital and JCAHO Pain assessment mandates – Power. Point & Computer Based Assessment – Competencies: • Patient Care • Medical Knowledge

Resident Satisfaction with the POA -Survey Response Rate 93% 2002 -2004 Question YES Have you learned any new clinical 70. 6% skills during this assessment? Do you think this was a useful way 83. 4% to spend part of orientation? Do you feel better prepared for some aspects of internship after this assessment? 80. 6% Do you expect to refer back to the teaching materials handed to you today? 84. 7% Would you recommend that we continue the POA next year? 84. 7%

The POA was based on the following… The University of Michigan Comprehensive Clinical Assessment (CCA)

• • Comprehensive Clinical Assessment 1991 -Present A high-stakes examination for University of Michigan Medical Students Fourth Year students must pass the CCA in order to Graduate 10 -15 stations over ~4 -5 hours ~150 M 4 students per year Summative Evaluation Standards have been set for Pass/Fail Remediation for failures Rochester AB. et. al. . Academic Radiology. 5(3): 169 -72, 1998

Summary of Station Comparison: CCA & POA- GME CCA-UGME • Formative • Summative • Informed Consent & Policy • History & Physical Diagnosis • Pain Assessment • Geriatrics • Socio-Cultural Communication • Evidence Based Medicine • Images

Institutional Implications for GME & UGME • Training the Faculty in the Teaching Skills of competency assessment – Medical School Objectives Project (MSOP) – Accreditation Council of Graduate Medical Education Reimbursement of faculty time and effort • Payment for both assessments – Medical School vs. Hospital • – Is the medical school or the hospital responsible for ‘educational cost’? • Trainee awareness of “competency gap” between the expectations of medical School and residency

POA Demographics 2002 -2004 Gender Male 54% Race White / Non-Hispanic Underrepresented Minorities Other 70. 1% 6. 5% 23. 4% Medical Schools Public 62% International United States Medical Licensing Examination (USMLE) 1. 5% Step 1 Step 2 231 233

The University of Michigan Health System Plan for Competency Based Resident Education • Joint Hire faculty member with the Department of Medical Education • Centralized OSCE – Post-Graduate Orientation Assessment (POA) This is based on the system already in place for Undergraduate Medical Education at UMHS.

The Department of Medical Education Chairman: Larry Gruppen, Ph. D • One of ~5 departments of medical education at LCME accredited medical schools • The department had focused on CME and UGME in the past with assessment and research expertise • The office of Graduate Medical Education and the Department of Medical Education – Joint Hire – Job Description • 50% appointment GME office, 50% appointment Dept. of Med. Ed. • Aid Residency programs with curriculum development, core competency assessment • Use this work as a vehicle of scholarly pursuit – The department has a long history of UGME – now will develop its expertise in GME