
f3869eeea54d37aed5a18e7b56d5d9bb.ppt
- Количество слайдов: 95
Nadim Lalani MD ONLINE EDUCATION IN THE ER
Vanilla Sky Tom Cruise 2001 Existential “Mind warp” Deals with cryogenics and the possibility of living a virtual life after death Blending of the technologic and biologic worlds “plugged in’ ? Virtual [technologic] world to supplement [real world] EM medical education
Objectives Definition Background Literature Review Med Ed Resident Ed Professional Development What it might look like Future Directions
What is “Online learning”? Online Learning [e-learning] = is digital Evolved from CD/computer labs Everyone does it! Performance Support [ for software e. g] Web page [e. g. Uptodate] Self-paced Web-based [CME] Leader-led [ Distance Learning] Blended [or hybrid] learning combines conventional with digital learning
Advantages of e-learning Rich environment: Media-filled [esp in EM] transfer of difficult concepts Links to sources Convenient, efficient & flexible Asynchronous Can be accessed from a distance Adult learning principles: Self-paced and self-directed Flexible/ home access efficiency
Background: Why bother? U of C Medical School current enrollment = 130 students goal 150 Mandatory EM rotation / increasing competencies Resident numbers also increasing Result more learners in the ED Relative shortage of preceptors, relevant clinical encounters and curricular time Will be worse when our program expands usurp learning opportunities
Why bother Deficiency in learning encounters = a performance gap future physicians do not have the adequate exposure to emergent problems. imperative we equip students, clerks & residents with the skills and training.
why bother Increased digitalisation is a key strategic goal of the U of C Learners are unique with mulitdimensional learning styles. Adult learning principles Attract the best candidates Provide a method of training students and clerks at two different campuses Provide consistency in teaching
Why bother? Provide efficient means of knowledge transfer to residents Increasing number of competencies [CANMEDs] Better use of academic half-day. Provide more effective professional development: Asynchronous don’t have to be there Interactive discussion board Consistent, evidence-based standard of practice Increased self-efficacy
E-learning not a panacea there is more to training and education than e -learning Certain skills do not lend themselves to elearning The key will be selecting the best delivery method. Cannot simply upload old material. Learner – focused no one solution [blended may work for residents].
Process: Can it be done? “Fail to prepare … prepare to fail” Need to address several key questions: 1. Purpose? Added value? 2. What support and expertise exist? 3. Ongoing upkeep? 4. Stakeholders? 5. Team? 6. Instructional design/Pedagogy
Literature Review Same Search terms in PUBMED Bibliographies of relevant articles scanned Missing 1 Med Ed & 1 CME [both foreign language]
Literature general Comments More Literature exists for Med Ed Pre 1990 Limited by lack of internal validity Few Randomised Controlled Studies Emerging Lit wrt Resident experience Despite lots of experience with online CME Little Literature … mostly Descriptive
Literature General Comments Terminology inconsistent Interventions vary. ? “prototypes” of today’s technology? Don’t address some of the uniqueness [internet] Comparing apples to oranges
E Learning & Med Ed Can E-learning be used to replace/augment Traditional Methods?
Study Dartmouth Med School 328 Students randomised to: Interactive Case-based study guide on Computer* Case-Based Printed study guide Anemia and Cardiology Courses Outcomes: Performance on higher order MCQ tests, exams Self-reported Efficiency * media-rich images, blood smears and EKG’s
Results Time Spent Cardiology Computer 4. 4 ± 0. 6 h Cardiology Workbook 9. 4 ± 0. 5 h Anemia Computer 5. 5 ± 0. 5 h Anemia Workbook 8. 0 ± 0. 5 h P = 0. 0001 P = 0. 001 No Difference in Test Scores No difference on board exams The vehicle is an acceptable means of delivery
Limitations Self reporting of efficiency! Confounders [other text books/practice exams/time-spent cramming] Doesn’t really tell us about dynamic problemsolving/ clinical judgment Board Exams Anemia CV Class + CAL 81. 5 ± 6. 5 80. 0 ± 6. 7 NS CAL 82. 7 ± 6. 0 82. 4 ± 5. 6 NS
179 Paeds Clerks in [2 sites Chicago] Randomised to Lecture via: Multimedia Text Book* Lecture Printed Text No intervention Paeds airway diseases Outomes: MCQ Test Score [at end of rotation & at 1 y later] * Only different in audio/video
Results Score / 26 MMTB Printed Book Lecture Control N = 89 21 19 23 26 Initial Test 16. 6 [2. 5] 16. 5 [3. 6] 15. 9 [2. 5] 14. 2 [2. 9] Final Test 15. 3 [2. 8] 15. 3 [2. 9] 14. 6 [2. 3] 14. 5 [3. 4]
Limitations 51% Attrition rate! Clerks at one site had mail-in repeat exam Confounders One hour instruction embedded in a 6 week clerkship
75 Med students [Brisbane Australia] Randomised after pretest to: Computer Tutorial Focus on knowledge Computer tutorial create ideal patient for dx + feedback [every 10 cases] Computer Tutorial both knowledge & decision + three different types of feedback [after every 10 cases] Looking at diagnosing abdo pain [ 30 cases]
Outcomes and Results Outcomes: Attained knowledge Diagnostic accuracy Decision-making confidence [self reported] Results: Students focusing on facts did not improve decision-making All feedback groups improved diagnostic accuracy Type of Feedback not important. Self reported confidence improved
Limitations Small study Very convoluted method ? reliability
E- Learning & Med Ed Can E-learning be used to Teach Procedural Skills?
82 Medical Students [Toronto & Augusta] Randomised to: Computer Tutorial + knot board Lecture with Feedback + knot board Two-Handed Knot tying Tested right after [filmed] Outcomes: Proportion Square/ Time to tie Knot Performance score [blinded surgeons] Student Questionnaire
Results NO difference in “Cognitive” portion Lower performance score in CAL group 89% Students would have preferred Lecture Session Lack of feedback cited as negative
Limitations Apples and Oranges! ? Not controlled for hands-on feedback Maybe CAL better if it described pitfalls / showed video of good and bad knots? Reliability of performance score [not included]
42 Clerks U of T Randomised to: Computer Tutorial [rich text, animations, interactive –Q&A, no audio/video] small group seminar [also interactive] Epistaxis Management Outcomes: Short Answer written Test Practical Test [16 point performance scale]
Results: Poor Prior knowledge No difference in written scores Slightly better practical skills with CBL
Limitations: Small numbers Examiners NOT blinded ? reliability of performance score [not included] Practical was on dummy ? transferability
69 Medical Students [Wisconsin] After pre-test Randomised to: Didactic Session/Q&A¥ Video-Tape Session* Computer Tutorial* Post Intervention: MCQ test, Practical Skills test [2 blinded obs] Repeat testing at one month ¥ no feedback. * Instructor present
Outcomes: MCQ Test Scores Timed observation of skills Critical Skills evaluated via checklist Performance Quotient calculated
Results Higher initial mean % correct / % complete in CBT group [p<0. 01] Significantly better PQ in CBT group at 1 month [p < 0. 01]
Results Didactic group better on immediate MCQ [63% vs 49% for video/CBT p < 0. 01] Difference in MCQ still there at 1 month
Results Bigger change in PQ with CBT at 1 month [ P< 0. 01]
Limitations Small study Video vs CBT essentially the same intervention ? Why CBT would do better than Video ? Reliability of checklist and PQ?
What About the ED Experience? Can E-Learning be used for Emergency Medicine Rotations?
100 Clerks [Mt Sinai] Randomised by blocks EM rotation with access to EM Website Modules [ACLS, Tox], Xrays, Pix, Paeds Cases EM Rotation without access Outomes: Exam Scores Student Satisfaction
Results ONLY 28% intervention group used it. 72% Cited lack of time NON sig difference in exam score [72. 8 vs 68. 2 p = 0. 058] Non sig difference in satisfaction [ 77. 5% vs 66% p = 0. 23] Baseline only 26% > 1 h /wk online [cf 96% next class] Baseline 65% wanted online component
Limitations < 30% in intervention group didn’t reach power. WAS ITT so results would have been +ve with more participants Problems with randomising by block rotations given away on lottery [ CARMs] Unmotivated learners? ? generalisable to clerks in 2008
23 Clerks [U of T Sick Kids] Volunteered for study, Randomised to Access to Web-based Modules No Access to Web-based modules ED Procedures [lac, LP, splint] Outcome: Performance on MCQ Test
Results Statistically higher competence [ p = 0. 0001] Cohen’s d Effect size r = 0. 79
Limitations Small sample size Volunteers [EM /techno gung-ho] Methodology: Unclear when test was administered in relation to rotation ? randomised to learning vs no learning? Validity of MCQ vs Observed skills Transfer of knowledge? MCQ vs Observed skills
350 Urology Clerks [4 med schools US] Randomised [two-group crossover] to: Web-based Tutorials [BPH, ED, PC, PSA] No Access to Web Tutorials Served as the controls for the modules they didn’t have access to online Outcomes: Performance on test [pre/post] [Cr =. 79] Durability of learning/ Learning efficiency in Sub. G
Results
Results:
Results Learning Efficiency 0. 10 vs 0. 03 [p<0. 001] Test scores still improved without WBT [12% BPH, 6% ED, 24% PC, 20% PSA] Web-based alone had Cohen’s r = 24. 9!
Limitations: Volunteers with unequal participation b/w sites [93% HMS vs 52% BUSM] High Drop out rate 210 /350 completed ? Generalisability of repeated measures ? Generalisability to EM
Summary: E-Learning & Med Ed Content can be delivered Appears to be transfer of learning of Cognitive skills … Perhaps also Psychomotor skills Still need for experienced clinician feedback Increased student satisfaction Attempt to make instructional design identical [validity] undermines uniqueness of EL Always will be apples and oranges
Summary E-Learning & EM: Controlled Interventional Literature conflicts Learners: Identify time constraints as a barrier in ER Want more visual aids [Ekg, XR, photo & WEB] Effective strategies for ER teaching include: Using Resources Going beyond patient care Improving the learning environment Adult principles/ learners may be driving force despite dearth of evidence
How Can we Augment Med Ed? Currently Clerks Use LMS “Osler” Download PDFs of Content/Cases Core Content Reviewed with Preceptor
We can do more … Procedures/ Anatomy Pictures Xray / Ekg interactive ppt
E Learning for Residents
109 IM Residents [US] during “clinics” year Randomised [Crossover] to: Access to Web-based Practice Guidelines On Web. CT [rich format with links to sources] Printed Practice Guidelines Outcomes: Format Preference Performance on Final test
Results Strong preference for Web Based material Men > Women NO difference on test between groups Non-Significant reduction in time spent ++ problems with WEbsite
Limitations Volunteer Significant dropout 145 eligible 109 enrolled 51 completed all aspects ? generalisability
162 FM & IM Residents Randomised to: Web-based tutorial [hyperlinks, graphics] Printed material ACC/AHA Guidelines Management post AMI Outomes: Test Score Efficiency Satisfaction
Results:
Limitations Only enrolled 30% [550 eligible] Participants not blinded to hypothesis ? Generalisable Voluntary + monetary honorarium
22 EM residents & Staff [U of T] Noninferiority for U/S vascular access course Randomised [after pretest] to: Web Tutorial + practice* 1 h Lecture + practice* Outcomes: [2 weeks later. . ] MCQ test 4 OSCE stations [blinded obs used checklist] *Non-precepted
Results Same pass rate All web users logged on from home
Limitations Small study [did have power though] No controlling of practice session Stronger coaching the weaker? No interrater reliability for OSCE
Summary: E-Learning + Residents Learning can be delivered this way Weak evidence of non-inferiority for learning guidelines in other specialties Modest evidence for use prepping for a hands -on session relating to EM. Weak evidence for Psychomotor skill acquisition Consistent satisfaction with the method
How to Augment Resident Ed Website / Subscriptions / Remerg. S HPS / AHD
Better Use of AHD … Longitudinal course on LMS? Already Licensed by U of C Used by Anesthesia
Interaction is key Learner-Sources, Learner-learner, Learner. Clinician
E Learning and CME
52 Physicians [US] randomised to: Web-Based CME Normal Instruction Office Dermatologic procedures Outcomes: Performance on test Satisfaction Self-reported performance change
Results: More improvement in test score with WBL [13. 2 vs 9. 6] General satisfaction with WBL Increased self-reported competency Fair amount of interaction in asynchronous forums
Limitations Small study Self-reporting ? Reliability of MCQ test ? Direct observation of procedures
99 Physicians [US] Randomised to: Web based CME for domestic violence Regular Instruction [only did two surveys] Outomes: Self-Efficacy Externally validated Survey instrument [CR 0. 7] Change in screening
Results: Increase in self efficacy + 18% intervention vs - 0. 6% control [p =0. 01] Positive other endpoints in survey NO difference in DV screening
Limitations Honoraria to WEB participants Methods ? No learning to controls? generalisability
103 physicians [US] randomised to: Web-based CME F-2 -F small group CME session Cholesterol management Outcomes: Performance on tests Chart audit [20 docs from each group 25 charts] Satisfaction survey
Results: More improvement with Online NO difference in Satifaction between groups Online spent 3. 8 h / 3 sessions Chart review statistically signif [small difference] in guideline adherence [but no difference in cholesterol screening]
Limitations 170 eligible only 103 included ? Different groups? [online better at baseline] Live CME event happened before Online [? cross contamination] Hard to reconcile change in behaviour when no change in screening ? generalisability
87 Physicians self-selected into: Online CME [3 sessions 8 h] F-2 -F CME [one 8 h session] Treatment of opioid dependence Outcomes: Post intervention test Satisfaction
Results NO difference in improvement No difference in satisfaction scores
Limitations 30% attrition [87 entered data for 62] NON-randomised ? reliability of MCQ test ? Generalisability
Summary: E-Learning and CME Certainly can be used to disseminate info In keeping with adult learning principles Transfer of cognitive component ? Psychomotor NO change in practice patterns Subjective increased self-efficacy
Enhancing CME Combine LMS with narrated slides Provide facilitation for your topic
Narrated Options for Quizzes
Future Directions International Emergency Medicine Disseminate info before experts hit the ground Collaboration with other programs Online Electives? EBM Toxicology Radiology
Thoughts?
Appendix: How you get there
Discerning the Context Learning Context Power Dynamics PGME Regionalisation People dynamics Resistance change Relatively few champions of Med Ed
Supporting Cast Stakeholders: University UME /PGME RMES Planning Committee: Content experts Educational specialists/Instructional design IT [graphic design/ media] End-users
Identifying Program Ideas Core content Literature Current material Needs analysis Narrow focus Not redundant
Sorting and Prioritising Prioritise content Important? Feasible?
Developing Objectives Needs to be competency based Reflect goals of UME/ Colleges
Instructional Plan How are you going to deliver this? Need interaction: Content, Instructors & peers Dedicated ED computer WEBSITE LMS
Instructional Plan Instructors: few innovators Clinicians Well suited “many hats” Need to “train the trainers” Learners: Self-directed & Computer savvy Clinical/ procedural Skills Blended approach
Instructional Plan Variety & content: Anatomy and radiology Power. Point, Flash files, video streaming. e-case modules LMS online discussion Logisitics: LMS powerful tool Free ones available STARS uses Moodle U of C has licenses & tech Support Regionalisation