A Mixed Reality Approach for Merging Abstract and

A Mixed Reality Approach for Merging Abstract and Concrete Knowledge John Quarles Department of CISE Samsun Lampotang Department of Anesthesiology Ira Fischler Department of Psycholgy Paul Fishwick Department of CISE Benjamin Lok Department of CISE

Anesthesia Education n Current Learning Process The Virtual Anesthesia Machine (VAM) – an abstract 2 D simulation A real Anesthesia Machine – a physical simulation

Knowledge Types Type Abstract Concrete Transfer Knowledge? Skill Set Concepts that can be applied to many real machine models Procedural, tactile, and psychomotor skills Example Invisible gas flow Performing a procedure with a specific machine

Problem: Merging Knowledge Types n Anesthesia machine trainees must mentally merge: ¨ Abstract Concepts ¨ Procedural Skills n Mixed Reality (MR) can merge real and virtual spaces ¨ Virtual space = Abstract ¨ Real Space = Concrete n Can MR’s Merging of spaces improve users’ merging of abstract and concrete knowledge?

Merging Abstract and Concrete n Merge the abstract physical simulation with the physical device

The Augmented Anesthesia Machine n Uses a magic lens approach to merge: ¨ Abstract VAM simulation ¨ Real Anesthesia Machine TUI

Overview The Augmented Anesthesia Machine n The Augmented Learning Process n User Study: VAM vs. AAM n n MR enables abstract + concrete knowledge merging

Previous Work n Tangible User Interfaces ¨ Two Main Parts: n Computational Media n Physical Interface ¨ Interface is meaningful n Representative of a part of the computational media n As opposed to an abstract interface (i. e. a mouse) ¨ AAM n Computational media = VAM n Physical Interface = real anesthesia machine [Ishii 1997, Ullmer 2000]

Previous Work n Magic Lenses ¨ 2 D to 3 D ¨ Applications in AR/MR n AAM ¨ Uses a magic lens to visually merge the abstract and concrete spaces [Bier 1993, Looser 2004]

Motivation n 75% of Anesthesia Machine Related Accidents resulting in death or brain damage are due to user error. ¨ Many anesthesia providers do not fully understand how the machine functions internally n VAM was created to reduce this number

Motivation for the VAM n n Created by Dr. Samsun Lampotang Advantages of the VAM ¨ visualize the invisible (i. e. gas flow) ¨ Spatial layout simplified for ease of visualization ¨ Web disseminated ¨ Increases understanding of internal function [Fishcler et. al. 2006] n n n 30000 Registered Users One Billion Hits per year (http: //vam. anest. ufl. edu) Problems still exist in knowledge transfer from the VAM to the real machine

Motivation n VAM to AAM: A Mapping Problem B A A B

MR: Enabling Multiple Represetations n Multiple Representations improve overall comprehension of a concept. VAM: Abstract Representation ¨ Anesthesia Machine: Concrete Representation ¨ AAM: Combined Representation ¨ n Proposed Augmented Learning Process: Abstract Representation (VAM) Combined Representations (AAM) Concrete Representation (Anesthesia Machine)

The AAM Representation n n Tracked 6 DOF magic lens shows overlaid VAM simulation Real machine interaction

Anesthesia Machine Tracking System n 2 D Optical Tracking with Open. CV ¨ Infrared Markers for knobs ¨ Infrared LEDs connected to buttons ¨ Color tracking for gauges (bright red) n 3 Unibrain webcams positioned around the machine in positions of minimal occlusion

Magic Lens Display n HP TC 1100 ¨ 3. 7 lbs ¨ 10” screen n Not See through ¨ Uses a 3 D model of the machine, registered to the real machine ¨ Easier to enable consistent registration ¨ Less hardware than video see through

Magic Lens Tracking System n n n Outside-Looking-in Optical Tracking Infrared Markers Specifications: ¨ 2 Unibrain Fire-I Webcams 640 x 480 at 30 fps ¨ Tracking Volume: 3 x 3 x 3 m ¨ Accuracy: 1 cm ¨ Jitter: 5 mm ¨ Latency: 70 ms

User Study Can AAM-Concrete help users to merge abstract and concrete knowledge? n Between subjects study (n=20) n ¨ VAM Training Group ¨ AAM Training Group

Population and Environment n 20 Psychology Students ¨ 4 males, 16 females ¨ Received class credit ¨ Knew nothing about anesthesia machines n Conducted in a quiet lab environment

Procedure Day 1 (90 min) 5 Training Exercises Informed Consent Introduction Machine Functions Day 2 (60 min) Self Evaluation of Training VAM AAM Spatial Cognition Tests Real Machine Intro Written Test VAM AAM Hands-on Fault Test Questionnaires

Metrics n Efficiency of Training ¨ Time n to complete 5 training exercises Abstract Knowledge Acquisition ¨ Written Anesthesia Machine Test n Short Answer and muliple choice n From the Anesthesia Patient Safety Foundation workbook n Concrete Knowledge Acquistion ¨ Hands-on fault test n Faulty inspiratory valve

Fault Test Results n Hypothesis 1: The AAM is more effective at teaching concrete concepts. Group AAM n 6 out of 10 VAM n # Participants Successful 1 out of 10 AAM group found the faults significantly more often than VAM group Accept Hypothesis 1: The AAM trains concrete knowledge more effectively than the VAM

Written Test Results Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts n There were no significant differences in Written test scores (p<0. 21) n

Training Time Results Hypothesis 2: The abstract representation of the VAM is more effective at teaching abstract concepts n VAM Group trained significantly faster (p<0. 002) n Accept Hypothesis 2: The VAM trains abstract knowledge more effectively than the AAM n

Discussion n Hypothesis 3: The AAM improves transfer to the real machine by enabling the merging of abstract and concrete knowledge. AAM group performed significantly better in the fault test To solve the fault test, participants had to merge: ¨ Concrete knowledge: notice the inspiratory valve was missing ¨ Abstract knowledge: understand the effect on the gas flow of the machine – harmful to the patient n Accept Hypothesis 3: AAM helps to merge abstract and concrete knowledge

Conclusions n leverage MR to merge simulation types ¨ Combining Abstract simulation with the corresponding physical device ¨ i. e. merging the VAM with the real machine MR enables multiple representations n MR can help users to merge abstract and concrete knowledge n ¨ Improves domains training transfer into real world