Model-Guided Therapy and the role of DICOM in

Model-Guided Therapy and the role of DICOM in Surgery Heinz U. Lemke, Ph. D Chair of Working Group 24 “DICOM in Surgery“

Content 1. Introduction (problems and solutions) 2. 3. 4. 5. Model guided therapy with TIMMS Classification and model classes Virtual human model examples Conclusion

Computer Assisted Digital OR Suite for Endoscopic MISS Problems: Multiple Data Sources Video Endoscopy Monitor Image Manager Report C-Arm Images MD’s Staff RN, Tech EEG Monitoring MRI Image PACS C-Arm Fluoroscopy Left side of OR Laser generator Image view boxes EMG Monitoring Digital endoscopic OR suite facilitates MISS Teleconferencing - telesurgery Courtesy of Dr. John Chiu

Model Guided Therapy and the Patient Specific Model • Model Guided Therapy (MGT) is a methodology complementing Image Guided Therapy (IGT) with additional vital patient-specific data. • It brings patient treatment closer to achieving a more precise diagnosis, a more accurate assessment of prognosis, as well as a more individualized planning, execution and validation of a specific therapy. • By definition, Model Guided Therapy is based on a Patient Specific Model (PSM) and allows for a patient specific intervention via an adapted therapeutic workflow.

Model Guided Therapy and data structures • Model Guided Therapy based on patient specific modelling requires appropriate IT architectures and data structures for its realisation. • For PSMs, archetypes and templates allow different levels of generalisation and specialisation, respectively.

Model Based Patient Care Modalities (X-ray, CT, US, MR, SPECT, PET, OI) Biosensors (physiology, metabolism, serum, tissue, …) Data bases (Atlas, P 2 P repositories, data grids, . . . ) Omics EMR Model Creation and Diagnosis (Data fusion, CAD, …) EBM Workflow IHE Mechatronics (Navigation, ablation, …) Model Maintenance and Intervention (Simulation, decision support, validation, …) IT Communication Infrastructure

Content 1. Introduction (problems and solutions) 2. Model guided therapy with TIMMS 3. 4. 5. 6. Classification and model classes Virtual human model examples PM data structures (SDTM and Open. EHR) Conclusion

Interventional Cockpit/SAS modules IT Model-Centric World View Repository Therapy Imaging and Model Management System (TIMMS) Images and signals Modelling tools IO Imaging and Biosensors Modelling Computing tools Simulation WF and K+D tools Rep. tools Kernel for WF and K+D Management Data Exch. Engine Visualisation Rep. Manager Control Devices/ Mechatr. tools Validation tools Intervention Validation Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Data and information Models (Simulated Objects) WF`s, EBM, ”cases” Models and intervention records

Model Guided Therapy with TIMMS • For a therapeutic intervention it is assumed that human, mechatronic, radiation or pharmaceutical agents interact with the model. • MGT provides the scientific basis for an accurate, transparent and reproducible intervention with the potential for validation and other services. • TIMMS is an IT meta architecture allowing for interoperability of the agents to facilitate a MGT intervention.

Model Guided Therapy The basic TIMMS patient model must have the following features: 1. 2. 3. 4. The TIMMS patient model must have components which represent the patient as an n-dimensional and multiscale (in space and time) data set. The TIMMS patient model must facilitate interfacing to the surgeon and other operative personnel, the TIMMS engines, TIMMS repositories, and the IT infrastructure. The TIMMS patient model must be capable of linking these components, which may be static or dynamic, in a meaningful and accurate way. For dynamic components, the TIMMS patient model must be able to process morphological and physiological data and perform the necessary mathematical functions to maintain the model in an up-to-date state.

Model Guided Therapy 5. 6. 7. 8. 9. The TIMMS patient model must be capable of being incorporated by the TIMMS executing workflow and responding to its changes. The TIMMS patient model must be amenable to be developed using readily available, standardized informatics methodology. Tools may include UML, XML, Visio, block diagrams, workflow diagrams, MATLAB, Simulink, DICOM (including surgical DICOM), Physiome, CDISC SDTM, open. EHR and similar products and tools. The TIMMS patient model must comply to software engineering criteria, for example, to open standards and service-oriented architectures to allow for multi-disciplinary information exchange. The TIMMS patient model must allow for further extensions to incorporate advances in molecular medical imaging, genomics, proteomics and epigenetics. The TIMMS patient model must be amenable to be used for clinical trials, predictive modeling, personal health records and in the long term contribute to a Model Based Medical Evidence (EBME) methodology.

Interventional Cockpit/SAS modules IT Model-Centric World View Repository Therapy Imaging and Model Management System (TIMMS) Images and signals Modelling tools IO Imaging and Biosensors Modelling Computing tools Simulation WF and K+D tools Rep. tools Kernel for WF and K+D Management Data Exch. Engine Visualisation Rep. Manager Control Devices/ Mechatr. tools Validation tools Intervention Validation Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Data and information Models (Simulated Objects) WF`s, EBM, ”cases” Models and intervention records

Generic and patient specific n-D modelling tools • • • Geometric modelling Prosthesis modelling Properties of cells and tissue Segmentation and reconstruction Biomechanics and damage Tissue growth Tissue shift Properties of biomaterials. . . Modelling tools

Model Guided Therapy • MGT in its simpliest instantiation is an intervention with a subset, a single or a set of voxels representing locations within the patient body. With this view, it is an extension from Image (pixel) Guided Therapy (IGT) to model (voxel) guided therapy. Examples of model guided therapy are: a) interventions within a subset of a voxel, e. g. cells, organelles, molecules, etc. b) interventions with a voxel, e. g. small tissue parts of an organ or lesion, etc. c) interventions with a set of voxels, e. g. part of functional structures of organs, organ components, soft tissue, lesions, etc.

Model Guided Therapy In a simple PSM, voxels may be associated with several dimensions of data 1. 2. 3. 4. 5. 6. 7. 8. 9. 1 -D signals (e. g. EEG) 2 -D projection and tomographic images 3 -D reconstructions Temporal change Tissue/cell type Ownership to organ, lesion, system, prothesis, chronic condition, etc. Spatial occupancy/extension Permeability (blood brain barrier) Flow (e. g. electric, heat, liquid, perfusion, diffusion, etc. )

Model Guided Therapy In a simple PSM, voxels may be associated with several dimensions of data 10. Level of oxygenation (e. g. level of hypoxia) 11. Pharmacokinetics (e. g. effect of tissue on pharmaceutical agent, flow parameters, time to peak, etc. ) 12. Pharmacodynamics (effect of pharmaceutical agent on tissue, ablation parameters) 13. Biological marker types (in vitro and/or in vivo molecular spectrum) 14. Reference coordinate system (e. g. Schaltenbrand/Warren, Talaraich/Tourneaux) 15. Value (life critical to life threatening) 16. Neighbourhood (e. g. 3³, 5³, 7³, etc. ) 17. .

Example: ENT model elements Source: G. Strauss

Example: ENT model elements Source: G. Strauss

Content 1. Introduction (problems and solutions) 2. Model guided therapy with TIMMS 3. Classification and model classes 4. Virtual human model examples 5. Conclusion

Strategies for multiscale modelling • Modelling is essential for understanding the knowledge of human characteristics such as, anatomy, physiology, metabolism, genomics, proteomics, pharmacokinetics, etc. • Because of the complexity of integrating the knowledge about the different characteristics the model of a human has to be realised on different levels (multiscale in space and time) and with different ontologies, depending on the questions posed answered delivered. • The problems associated with using reduced-form components within large systems models stem primarily from their limited range of validity.

Source: J. Bassingthwaighte

Patient specific and associated modelling functions In the Model-Centric World View a wide variety of information, relating to the patient, can be integrated with the images and their derivatives, providing a more comprehensive and robust view of the patient. By default, the broader the spectrum of different types of interventional/surgical workflows which have to be considered, the more effort has to be given for designing appropriate multiscale PSM’s and associated services.

Patient specific and associated modelling functions Management of n-D and multi resolutional knowledge (model of the biologic continuum in space and time) is still a research and development challenge. If solved successfully, it will transform surgery into a more scientifically based activity.

Content 1. Introduction (problems and solutions) 2. Model guided therapy with TIMMS 3. Classification and model classes 4. Virtual human model examples 5. Conclusion

Patient Specific CMB Human Laser Scan (CAESAR DB) Multimodal Imaging (MRI, CT, Angio, . . DT-MRI) Visible Human Anatomical Template organ surface meshes Roberts JHU Spitzer 2006 Virtual Anatomy PKPD FEM Mesh (Roberts JHU)

Content 1. 2. 3. 4. Introduction (problems and solutions) Model guided therapy with TIMMS Classification and model classes Virtual human model examples 5. Conclusion

Solutions and Research Focus (medical) • Transition from image guided to model guided therapy (e. g. through workflow and use case selection/creation/repositories) • Concepts and specification of patient specific models in a multiscale domain of discourse • Concepts and design of a canonical set of low level surgical functions • Prototyping

Interventional Cockpit/SAS modules IT Model-Centric World View Repository Therapy Imaging and Model Management System (TIMMS) Data Exch. Engine Control Prototyping Images and signals Modelling tools IO Imaging and Biosensors Modelling Computing tools Simulation WF and K+D tools Rep. tools Kernel for WF and K+D Management Visualisation Rep. Manager Devices/ Mechatr. tools Validation tools Intervention Validation Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Data and information Models (Simulated Objects) WF`s, EBM, ”cases” Models and intervention records

Solutions and Research Focus (technical) • Concepts and data structure design of patient specific models (e. g. with archetypes and templates) • Model management with open architectures (e. g. SOA) • SOA modulariation with repositories, engines, LLM´s and HLM´s • LLM´s as adaptive (cognitive/intelligent) agents • HLM´s as application modules (competitive differentiation) • LLM´s possibly as open source • Kernel (engine and repository) for adaptive workflow and K+D management • Cooperative and competitive R+D framework for engine and repository building • Therapy based open standard ( e. g. S-DICOM) • Transition from CAD to CAT modelling

Interventional Cockpit/SAS modules IT Model-Centric World View Repository Therapy Imaging and Model Management System (TIMMS) Data Exch. Engine Control Archetypes and Templates Images and signals Modelling tools IO Imaging and Biosensors Modelling Computing tools Simulation WF and K+D tools Rep. tools Kernel for WF and K+D Management Visualisation Rep. Manager Devices/ Mechatr. tools Validation tools Intervention Validation Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Data and information Models (Simulated Objects) WF`s, EBM, ”cases” Models and intervention records

Solutions and Research Focus (medical and technical) • Transition from image guided to model guided therapy (e. g. through workflow and use case selection/creation/repositories) • Use cases for adaptive workflow, exception handling and K+D management for selected interventions • Cooperative and competitive R+D framework for low (open source) and high level (competitive differentiation) surgical function computerisation • Information/model flow from diagnosis (e. g. CAD) to CAT (i. e. interdisciplinary cooperation) • Development of standards for patient modelling in WG 24 “DICOM in Surgery”

Interventional Cockpit/SAS modules IT Model-Centric World View Repository Candidate components for open source Data Exch. Engine Control Open Source Images and signals Modelling tools IO Imaging and Biosensors Modelling Computing tools Simulation WF and K+D tools Rep. tools Kernel for WF and K+D Management Visualisation Rep. Manager Devices/ Mechatr. tools Validation tools Intervention Validation Therapy Imaging and Model Management System (TIMMS) ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy Data and information Models (Simulated Objects) WF`s, EBM, ”cases” Models and intervention records

WG 24 “DICOM in Surgery“ Project Groups • • • PG 1 PG 2 PG 3 PG 4 PG 5 PG 6 PG 7 PG 8 PG 9 PG 10 PG 11 WF/MI Neurosurgery WF/MI ENT and CMF Surgery WF/MI Orthopaedic Surgery WF/MI Cardiovascular Surgery WF/MI Thoraco-abdominal Surgery WF/MI Interventional Radiology WF/MI Anaesthesia S-PACS Functions WFMS Tools Image Processing and Display Ultrasound in Surgery

Definition of Surgical Workflows (S-WFs) • Micro Laryngeal Surgery (MLS) (PG 2 ENT/CMF) • Foreign Body Excision (PG 2 ENT/CMF) • Total Hip Replacement Surgery (PG 3 Orthopaedic) • Total Endoscopic Coronary Artery Bypass (TECAB) (PG 4 Cardiovascular) • Mitral Valve Reconstruction (MVR) (PG 4 Cardiovascular) • Laparoscopic Splenectomy (PG 5 Thoraco-abdominal) • Laparoscopic Cholecystectomy (PG 5 Thoraco-abdominal) • Laparoscopic Nephrectomy left (PG 5 Thoraco-abdominal) • Angiography with PTA and Stent (PG 6 Interventional Radiology) • Hepatic Tumor Radio Frequency Ablation (PG 6 Interventional Radiology) • Trajugular Intrahepatic Portosystemic Shunt (PG 6 Interventional Radiology)

CARS 2008 Computer Assisted Radiology and Surgery CARS / SPIE / Euro. PACS 9 th Joint Workshop on Surgical PACS and the Digital Operating Room Barcelona, 28 June, 2008 12 th Meeting of the DICOM Working Group WG 24 “DICOM in Surgery“ Barcelona, 28 June 2008 http: //www. cars-int. org

WG 24 “DICOM in Surgery” Secretariat: Howard Clark, NEMA Secretary: Franziska Schweikert, CARS/CURAC Office fschweikert@cars-int. org General Chair: Heinz U. Lemke, ISCAS/CURAC, Germany Co-Chair: Ferenc Jolesz, Harvard Medical School, Boston (Surgery/Radiology) Co-Chair: tbd (Industry)