Speaking of Virtual Patients What are they really

Speaking of Virtual Patients: What are they, really? A reasonable question? ? ? Thomas B. Talbot, MD, MS, FAAP Med. VR Group USC Institute for Creative Technologies

Let’s talk about what a virtual patient is…… § Let’s talk about what a virtual patient is § Virtual Patients we can measure (standards based) – Case Presentations – Interactive Patient Scenarios § Virtual Patients that don’t fit – – – § 2 Virtual Patient Games High Fidelity Software Simulations High Fidelity Manikin Simulators Human Standardized Patients Dialog Agents & Virtual Standardized Patients The future of Virtual Standardized Patients

CASE PRESENTATION 3

CASE PRESENTATION 4

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated Interactivity CASE PRESENTATION Online Virtual Patient Basic Knowledge, Clinical Reasoning Knowledge, Pattern Recognition Very Low Consistency of Experience & Evaluation High Flexibility to Recover from Learner Errors None Suitable for Game Approach? None Authoring Challenge Low Core Technology 5 Enabling Technology Text, Multimedia HTML, Authoring Tools

INTERACTIVE PATIENT SCENARIO 6

7

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated Interactivity INTERACTIVE PATIENT SCENARIO Virtual Patient, Multimedia Sim, Branching Scenario Clinical Reasoning Case Knowledge, Clinical Reasoning Low-Moderate Consistency of Experience & Evaluation High Flexibility to Recover from Learner Errors Low Suitable for Game Approach? Moderate Authoring Challenge Moderate Core Technology 8 Enabling Technology Branching, Multimedia HTML, C, Actionscript, Multimedia Authoring Tools

INOTS 9

VIRTUAL PATIENT GAME 10

Type of Virtual Patient Common Names VIRTUAL PATIENT GAME VR Simulation, Virtual Patient Avatar Teaching Applications Procedures, high-risk scenarios, team training Learner Skills Evaluated Clinical & Procedural Reasoning, Physical Diagnosis Interactivity Moderate-High Consistency of Experience & Evaluation Moderate Flexibility to Recover from Learner Errors Moderate Suitable for Game Approach? High Authoring Challenge High Core Technology 11 Enabling Technology State-Machines, Branching Game Engine

High Fidelity Software Simulation 12

13

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated HIGH FIDELITY SOFTWARE SIMULATIONS Medical Simulation, VR Surgical Simulator Complex Procedures, Surgeries, High-risk Scenarios Procedural Skills, Critical Interventions Interactivity High Consistency of Experience & Evaluation High Flexibility to Recover from Learner Errors High Suitable for Game Approach? Moderate Authoring Challenge Very High Core Technology 14 Enabling Technology Physiology Engine, Virtual Reality Tissue Model Game Engine, Haptics, Anatomy Data

15

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated Interactivity HIGH FIDELITY MANIKINS Manikin, Task Trainer Procedures, Team Training, Codes Clinical Protocols, Physical Diagnosis, Team Performance Moderate Consistency of Experience & Evaluation Low-Moderate Flexibility to Recover from Learner Errors Moderate Suitable for Game Approach? Authoring Challenge Core Technology 16 Enabling Technology Low Moderate Varies Microcontrollers, sensors, actuators, wireless data

HUMAN STANDARDIZED PATIENTS 17

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated HUMAN STANDARDIZED PATIENTS Standardized Patient, Human Actor Patient History, Physical Exam, Patient Counseling Interview Skills, Physical Diagnosis Interactivity Very High Consistency of Experience & Evaluation Very Low Flexibility to Recover from Learner Errors High Suitable for Game Approach? None Authoring Challenge Low Core Technology None 18 Enabling Technology Written Script, Video Recorder, Human Monitor

19

Type of Virtual Patient Common Names Teaching Applications Learner Skills Evaluated Interactivity VIRTUAL STANDARDIZED PATIENTS Virtual Human, Conversational Agent Patient History, Patient Counseling Interview Skills Very High Consistency of Experience & Evaluation High Flexibility to Recover from Learner Errors High Suitable for Game Approach? High Authoring Challenge Very High Core Technology Dialogue AI, Natural Language Processing 20 Enabling Technology 3 D Game Engine, Speech, Non-verbal behavior generation, Speech Recognition

Video Time 21

Simcoach Standard Patient Studio § $8 M of Do. D Advanced Technology R&D Project § A large public trial and distribution of the technology over 4 years § Ambitious scope, yet uses mature technology to mitigate risk § Usable, refined tools for the public – Free – Goal is to create a “critical mass” with the medical education community § Serious emphasis on student assessment § Designed with ‘lessons learned’ from prior work in mind § Desire to replicate the “gold standard” or surpass it 22

What it is: § Conversational Virtual Standardized Patient (The “Clinic”) – Includes plethora of ICT technology: § Avatars, Natural Language Understanding, Artificial Intelligence Dialogue System, Automated Non-Verbal Behavior Generation, Emotional Expression, Trust-based Responses – Works through web-browsers & tablets § Online Authoring System (The Studio) – Guided authoring system targeting medical educators § Leverages prepared ‘personalities’ that are modified § Learner ‘questions’ are mostly pre-packaged because they are the most difficult aspect of patient authoring § Two-phase “open questioning” / “review of system” model § Simple assessment authoring integrated into tools – Shared assets with forum and user rating system 23

The Standard Patient Studio Team § ICT Teams – Med. VR Group § § Rizzo, Talbot, Williams and others Screenwriting Team – Simcoach Group § Eric Forbell – Integrated Virtual Human Team § § Arno Hartholt Includes Art Team – Assessment Group § Chad Lane § External Partners – Breakaway Ltd, § – USC Keck School of Medicine § Dr. Win May and others – Uniformed Services University School of Medicine § § National Capitol Area Simulation Center Gil Muniz, Alan Liu, et al. – National Board of Medical Examiners § § 24 Serious game development company, Timonium, MD Steve Clyman Galen Buckwalter

Human Standardized Patients § The “Gold Standard” dialogue-based training system – Rely on student initiative to drive encounter dialogue, ask questions, dynamically follow leads – Close approximation to actual patient encounters – Genuine physical exam and findings § Disadvantages – – – § High verbal, but low factual accuracy Expensive, difficult retention Some pathologies & ages unavailable Few opportunities to access them Uncontrolled variation, subjective assessment There is no desire to replace or eliminate HSPs – HSPs have inherent advantages 25

Lessons from the past regarding Virtual Patients Lack Dialogue Are inflexible Not Readily Authorable Lack of content ‘critical mass’ Emphasize things that computer sims aren’t good for Target populations that do not need the training Lack critical interactive technologies Too expensive, too proprietary Avatar is often superfluous Do not emulate gold-standard approach Lack automated & well designed assessment Lack breadth 26 Attempted things that were too complicated

SPS: Achieving A Gold Standard Interaction 1. Reading the chart 2. The patient interview (VSP session) 1. 2. 3. 4. 5. 3. Greeting the patient Obtaining the chief complaint Establishing Rapport Open questioning Specific questioning (Review of Systems) Patient Examination (single text page or VSP session) 1. 2. Physical Examination Review Laboratory or Radiology information 4. Make a Diagnosis and Treatment Plan (single menu page) 5. Counsel the patient on the diagnosis and treatment plan (VSP session) 6. Assessment (visual summary of skills and progress) 1. 2. 27 Errors of commission and omission System’s estimate of progress on learning skills and sub-skills GREY: Lower Emphasis Areas BLACK: Higher Emphasis Areas

A Variety of Encounter Types 28

The Patient Interview § Virtual Human Encounter – Full ICT Virtual Human Capabilities – Typed Input, Verbal & Nonverbal Output § Natural Language Recognition § Speech recognition year 3 § On-Screen Educational Guides – Assessment Indicators § Rapport, Achievements § Not used for evaluation oriented patients – Virtual Attending Physician (VAP) § Automatically asks a question when pressed § Conversation – Free-text Question/Answer model § Patient may ask a multiple choice question – Open to Closed Questioning – Answers rapport dependent 29

Virtual Standardized Patient Interview Details § Based on pre-authored “Personalities” – Will create a very well-built out patient with negative/normal responses to a wide variety of medical questions – At least five personality variants will be created – Most educator-authored patients will be based off these personalities § Variety of appearances – Educator will be able to select sex, face, skin, habitus, clothing, etc – Non-verbal tone and baseline rapport is selectable – Respiratory rate/depth can be set or based upon physiology engine data at authoring § Guided authoring by medical educators – First, enter chief complaint, diagnosis – Second, enter patient descriptive dialoge in response to open-ended questioning – Third, select items from “Review of Systems” and history lists, alter responses for appropriate items and select items for evaluation scoring 30

Patient Interview Editor (Open Phase) 31

32

Virtual Attending Physician (VAP) § A Virtual Human Encounter – White Coat Attending – Simulated Socratic Session § Multiple-choice INOTS-like encounter – Choices with consequences – Unique feedback based upon learner response § Multiple Uses – Patient diagnosis & evaluation – Teaching points – Test for understanding 33

Physical Examination Phase § Different than other physical exams – Features are resource constrained – Will not attempt to do things that can now be done better via other means. § Multimedia interface – Pictures, sounds, videos or animations can be loaded – Think “X-Rays, rashes, auscultation, etc” – Lab/rad results available (physiology engine derived or author entered) § Available via menu or command line § Unique Virtual Human Encounter – Non-verbal, non-dialogue – Responds to commands to perform maneuvers § Neurological & Musculoskeletal focused § “Stand on one leg and close your eyes” – Easy “check the box” based authoring 34

Patient Diagnosis & Ordering § Simple One Page Multimedia Form – Select diagnosis from choices – Select options to order for patient plan § Auto-populates labs/radiology from physical exam phase 35

Patient Counseling § Virtual Human Session – Dialogue Based, Multiple-Choice or both? (undecided) – Resource Constrained – Optional § What the learner should do – Explain the diagnosis & treatment plan – Answer questions – Assess for patient understanding 36

Assessment Screen: Robust & Graphical – Interview Phase § § § – Rapport & Trust Open/Closed question ratio Must Ask Items (critical) Nice to have asked items (secondary) Acts of commission (bad things you said) Efficiency VAP Encounter § Score-based feedback or no Feedback – Physical Examination Phase Critical & secondary items selected § Efficiency score § – Counseling Phase § Undetermined (limited assessment in this version) – Diagnosis & Treatment § 37 Direct feedback on selections made vs. optimal

Initial Use Cases: 38 “Bread & Butter” Medical Student / Intern Cases

Development Plan § Development Phase (18 -24 months) – – § Public Use Phase (24 months) – – § 39 Construct tools, interface Build prototype baseline-personality Initial cases built & secondary personalities authored Research-based question database optimization Open to second-level partners for first three months Open publically Public case authoring, student use, forum community Public presentations & seminars at national events Reporting

Research Questions § Standard Patient Encounters – How useful are virtual standardized patient encounters? – What are the comparative advantages and disadvantages of VSPs compared to human standardized patients? – What kinds of patients, diagnoses or other characteristics are best/worst for VSP encounters? – What phase of the clinical encounter is preferred by educators for a VSP encounter? – How do students respond to the presence of a virtual attending physician (VAP) quizzing and guiding them? Is it useful? – How can VSP encounters best be blended with live standardized patients? § What are the cost implications? § How can use of VSPs free up human resources to expand availability of human standardized patients (HSPs)? If so, what will HSPs being doing more of? 40

Research Questions § Capabilities – What capabilities are in greatest practical use? – What capabilities are not being used? Why? – What are the additional capabilities desired by the medical education community? § Other Results – – 41 Number of cases and patients authored Student assessment and performance data Author and student feedback Project impact results

Research Questions § Authoring – Can medical educators successfully author compelling VSP interactions when this formerly required teams of experts? – What are the unexpected uses and author populations for this technology? Nurses, allied health, dentists, pharmacists, sales, customer service? – What are the limitations for educator authored VSPs? § What features requires more sophisticated authors? § How can authoring be improved to extend these domains to less sophisticated authors? § Assessment – Are VSP encounter assessments equal or superior to traditional HSP encounter assessments? How? – Is there value to offering a third person perspective replay? – How can an automated assessment be successful in graphically depicting a concise assessment of student performance that has utility for guiding future student performance? – How does VSP encounter performance change with repetition? What is the optimal number of repetitions? – Are VSPs and the assessment good enough for use in state-mandated testing? 42

STANDARD PATIENT STUDIO SUMMARY § The Simcoach Standard Patient Studio uses ICT’s unparalleled technology § The impact of SPS will be revolutionary § SPS will be a special national resource § SPS will closely approximate HSP encounters, but with some advantages § SPS will be useful for both learning & evaluation § SPS will provide objective student data 43

CONCLUDING COMMENTS § There are lots of different things called ‘Virtual Patients” § Various technologies tend to be stronger for specific types of medical training. Choose accordingly. § Use the easiest technology that achieves the learning objective § The technology is a tool, not an end goal in itself § Assessment is really important § Newer types don’t fit into the Medbiquitous model, but we can probably find practical solutions for this. 44