IGT System Engineering Peter Kazanzides Assistant Research Professor

IGT System Engineering Peter Kazanzides Assistant Research Professor Department of Computer Science Johns Hopkins University October 19, 2006 Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

My Background 1989 -1990 Postdoctoral research at IBM on ROBODOC 1990 -2002 Co-Founder of Integrated Surgical Systems – Commercial development of ROBODOC® System – Commercial sales in Europe (CE Mark) – Clinical trials in U. S. and Japan 2002 -present Research faculty at JHU CISST ERC – Lead engineering infrastructure – Develop image-guided robots Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Outline • What is System Engineering? • Major activities: – – – Requirements Risk Analysis Architecture Modeling / Simulation Verification and Validation • Case study: Image-guided robot for rodent research • Current work: Surgical Assistant Workstation • Summary • Three challenges and opportunities for assistance Four Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

What is System Engineering? • Spans the entire development process • Considers the entire system, including hardware and software (interdisciplinary) INCOSE: International Council on Systems Engineering (www. incose. org) The SIMILAR Process Customer Needs State the Problem Investigate Alternatives Model the system Integrate Launch System Performance Re-evaluate Re-evaluate Assess Outpu t The Systems Engineering Process from A. T. Bahill and B. Gissing, “Re-evaluating systems engineering concepts using systems thinking”, IEEE Transaction on Systems, Man and Cybernetics, Part C: Applications and Reviews, 28 (4), 516 -527, 1998. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

System Engineering Activities • • • Requirements Risk analysis Architecture Modeling / simulation Verification and Validation • Level of effort will vary based on factors such as: – University or industry project – Plans for clinical evaluation – Cohesiveness of developers and users Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Requirements • Informal (undocumented) is fine for early prototypes • Documented requirements necessary for: – Any system for clinical use – Any development involving multiple/distributed parties (e. g. , university researchers, industry, clinicians) • Requirements may not be necessary for toolkits Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Risk Analysis • Should be performed by cross-functional team, including application expert • Failure Mode Effects (and Criticality) Analysis (FMEA/FMECA) – Bottom up analysis: for each component failure, determine (potential) system failure – Most useful in design phase (proactive) • Fault Tree Analysis (FTA) – Top down analysis: trace each system failure down to components – Most useful for after-the-fact analysis Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Architecture • Often refers to software, but can include hardware elements • Simple definition: “What the pieces are and how they work together” • From Software Engineering Institute (SEI) at CMU: Software architecture is the set of design decisions which, if made incorrectly, may cause your project to be cancelled – Eoin Wood Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Architecture: 4+1 View From: P. Kruchten, “The 4+1 View Model of Architecture”, IEEE Software, 12(6), Nov 1995. Other approaches include RM-ODP, Zachman framework Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Modeling and Simulation • Create models of system to guide development • Model Driven Architecture (MDA) ®* – – Model is enduring asset Perform simulation/testing with model Generate code from model Is the technology (tool set) there yet? “Processes, Methodologies, and Tools used for the Development of a Model Driven Architecture Based Open Software Framework for Distributed Medical Devices”, Amen Ra Mashariki, Ph. D. proposal, Morgan State University *Registered trademark of OMG Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Verification and Validation (V&V) • NASA SATC: “differences between verification and validation are unimportant except to theorist” • Two primary V&V activities: – Reviews (inspections, walkthroughs) – Testing (e. g. , against requirements) Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Case Study: Image-Guided Robot for Rodent Research Initial application: correlate p. O 2 measurements with PET values to validate non-invasive method for locating hypoxic tumor regions PET pixel value Copyright © CISST ERC, 2006 p. O 2 probe reading NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Image-Guided Robot for Rodent Research: Requirements • Distributed team: – Developers at JHU (Baltimore) – Customers (users) at Memorial Sloan Kettering Cancer Center (New York City) • Requirement were critical: – – First meeting: Sept 2003 Three major revisions Final version approved: March 2004 System installed at MSKCC: Jan 2005 Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Image-Guided Robot for Rodent Research: Physical Architecture PC (Windows) 3 D Slicer Copyright © CISST ERC, 2006 Servo Control and Amps (Galil) Ethernet DMC-2143 Robot NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Image-Guided Robot for Rodent Research: Development Architecture Application Environment File I/O Registration Executable Testing Environment File I/O CISST LAPACK Region Growing GUI vtk. Rodent TCL/TK Interpreter Python Interpreter (IRE) mskcc. Robot 3 D Slicer VTK wx. Python Galil driver Open source Application-specific Proprietary to vendor Copyright © CISST ERC, 2006 Test Scripts CISST wx. Widgets Ethernet NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

CISST Software Package Foundation libraries cisst. Common cisst. Vector cisst. Numerical cisst. Interactive Open Source www. cisst. org/cisst Real Time Support cisst. OSAbstraction cisst. Device. Interface cisst. Realtime Interactive Research Environment (IRE) Interventional Devices cisst. Stereo. Vision cisst. Tracker cisst. Robot … Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Image-Guided Robot for Rodent Research: Verification & Validation • Well-tested toolkits (Slicer, VTK, CISST, …) • System accuracy tests with phantom: – PET Fiducial Localization Error: 0. 26 mm – Robot Fiducial Localization Error: 0. 18 mm – Target Registration Error: 0. 29 mm • Customer acceptance testing “Design and Validation of an Image-Guided Robot for Small Animal Research, ” MICCAI, Copenhagen, Denmark, Oct 2006. Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Microscopes Endoscopes Ultrasound Surgical Assistant Workstation ISI License Intuitive Comm. Interface Intuitive API JHU Robot Interfaces API Emulator CISST – ISI “wrapper” Research Applications and Subsystems Copyright © CISST ERC, 2006 HMD; Stereo TV • • • Optimization Virtual Fixtures Stereo processing Tool tracking Image registration Interactive visualization Ultrasound Video Visualization Subsystem • • • Haptics Task modeling Skill Assessment Remote telesurgery Etc. NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Surgical Assistant Workstation • NSF supplement to CISST ERC – Started Sept 2006 • Collaborative effort between JHU and Intuitive Surgical • Currently working on formal specifications – Functional specification – Architecture views (4+1) Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Summary • System engineering integrates multiple disciplines over the development life cycle • Key activities include requirements, modeling, architecture, verification & validation – What’s needed depends on development scenario • But, there are challenges in all of these activities… Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #1: HW and SW for IGT Systems Input Device(s) Preoperative Data and Models Patient Database Electronic Records Preoperative Images Application Controller System Configuration HL 7 DICOM Interfaces / Middleware Procedural Logic (e. g. , Task Graph) Data Logging Anatomical Atlas DICOM Intraoperative Imaging Visualization/GUI (e. g. , X-ray, ultrasound, microscope video) Image Acquisition Medical Robot Trajectory Control Visualization/GUI (Navigation System) Servo Control Target Positions Image Processing Display(s) Image Overlay Navigation or Tracking System Real-Time Data Distribution Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #1: HW and SW for IGT Systems • Intraoperative imaging – Hardware exists (X-ray, US, video, …, CT, MRI) – Need better data access (e. g. , US RF) • Navigation/tracking – Hardware exists – Software toolkits recently available – Research interfaces to commercial systems emerging • Robots – Mostly custom hardware – Software toolkits not yet available – Research interface to da. Vinci emerging • Application control / Workflow – Software toolkits available (Slicer, SIGN, IGSTK, …) Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #2: Architecture and Interface Standards Input Device(s) Preoperative Data and Models Patient Database Electronic Records Preoperative Images Application Controller System Configuration Interfaces / Middleware DICOM Intraoperative Imaging Visualization/GUI (e. g. , X-ray, ultrasound, microscope video) Image Acquisition Medical Robot Standards needed Copyright © CISST ERC, 2006 Image Overlay Navigation or Tracking System Trajectory Control Visualization/GUI (Navigation System) Servo Control Target Positions Image Processing Display(s) Standards needed HL 7 DICOM Procedural Logic (e. g. , Task Graph) Data Logging Anatomical Atlas Real-Time Data Distribution NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #2: Architecture and Interface Standards • Need standards for: – Technical interoperability: delivery of messages/data between subsystems (e. g. , networks, middleware) – Semantic interoperability: ability to process messages (e. g. , common data structures, commands) • Existing efforts: – – DICOM for medical images ISO 11073 (IEEE 1073) CANOpen Medical device plug & play (CIMIT) • Examples of tracking system interfaces: – – SIGN / Open. Tracker IGSTK / Atamai cisst. Tracker / cis. Tracker (JHU) Orion (Vanderbilt) Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #3: Tools for modeling, simulation, and testing • Can we predict system performance before implementation? • Can we do automated nightly testing of software that interacts with the real world? – Are simulations good enough? Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Challenge #4: System engineering in a research environment • Define a System Engineering process that: – Facilitates clinical translation – Enables efficient interactions with industry – Does not delay the pace of research Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

How can NCIGT (or ? ? ? ) help? 1. Support database of IGT hardware and software solutions 2. Promote the development and adoption of standards for IGT 3. Develop realistic modeling and simulation environments for system evaluation and testing 4. Provide guidance documents for performing system engineering Copyright © CISST ERC, 2006 NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology