CAS clinical applications CAS Srping

CAS clinical applications • • • CAS, Srping 2002 Neurosurgery Orthopaedics Maxillofacial, craneofacial, and dental surgery Laparoscopic and endoscopic surgeries Radiotherapy Specific procedures in ophtalmology, othorhinolaringology, etc. © L. Joskowicz 3

Elements of CAS systems CAS, Srping 2002 © L. Joskowicz 4

Technical elements of CAS systems 1. Medical images 2. Medical image visualization 3. Segmentation and modeling 4. Virtual and augmented reality, tele-surgery 5. Preoperative analysis and planning 6. Image and robot registration 7. Medical mechanical and robotics systems 8. Real-time tracking 9. Safety, man-machine interface, human factors CAS, Srping 2002 © L. Joskowicz 5

Key parameters for understanding and comparing solutions • How many procedures are performed yearly? • What is the rate of complications? What are their causes? • In what aspects can a CAS system help? • Does it address part of a clinically important problem? • What stage is the system in: in-vitro, cadaver, clinical trials? CAS, Srping 2002 © L. Joskowicz 6

1. Medical Images CAS, Srping 2002 © L. Joskowicz

Most common imaging modalities • Film X-ray, Digital X-ray, Fluoroscopy, Digital Substraction Angiography (DSA) • Ultrasound -- 2 D and 2. 5 D (stack of slices) • Computed Tomography (CT) • Magnetic Resonance Imaging (MRI) • Nuclear Medicine (NM) – PET -- Positron Emission Tomography – SPECT -- Single Photon Emission Tomography CAS, Srping 2002 © L. Joskowicz 8

Medical images: characteristics (1) • Preoperative or intraoperative use – depends on the size and location of imaging machine • Dimensionality: 2 D, 2. 5 D, 2 D+time – projection, cross section, stack of projections, time sequence • Image quality – pixel intensity and spatial resolution – amount of noise; signal/noise ratio – spatial distortions and intensity bias CAS, Srping 2002 © L. Joskowicz 9

Medical images: characteristics (2) • Field of view • Radiation to patient and to surgeon • Functional or anatomical imaging – neurological activity, blood flow, cardiac activity • What it’s best at for – bone, soft tissue, fetus, surface/deep tumors, etc • Clinical use – diagnosis, surgical, navigation, CAS, Srping 2002 © L. Joskowicz 10

X-ray images • Measure absorption of x-ray radiation from source to set of receptors • Film X-ray has very high resolution Gray value proportional to radiation energy CAS, Srping 2002 © L. Joskowicz 11

X-ray Fluoroscopy CAS, Srping 2002 © L. Joskowicz 12

Fluoroscopic images CAS, Srping 2002 © L. Joskowicz 13

X-ray image properties • Traditional, cheap, widely available • Two-dimensional projections (at least two required) • High resolution, low noise (more fluoroscope) – film size, 64 K gray levels – fluoroscopic images: TV quality, 20 cm field of view • Relatively low radiation • Bone and metal images very well • Fluoroscopy used for intraoperative navigation CAS, Srping 2002 © L. Joskowicz 14

Ultrasound imaging (US) • Measure refraction properties of an ultrasound wave as it hits tissue • No radiation • Poor resolution, distortion, noise • Low penetration properties • One 2 D slice or several slices (2. 5 D) • Relatively cheap and easy to use • Preoperative and intraoperative use CAS, Srping 2002 © L. Joskowicz 15

Ultrasound imaging CAS, Srping 2002 © L. Joskowicz 16

Computed Tomography (CT) CAS, Srping 2002 © L. Joskowicz 17

Computed Tomography Images cuts d = 5 mm d = 25 mm CAS, Srping 2002 © L. Joskowicz d = 15 mm d = 35 mm 18

Computed Tomography Principle X-rays intensity angle CAS, Srping 2002 © L. Joskowicz 19

Computed Tomography Properties • Sepcifications: – 512 x 512 12 bit gray level images; pixel size 0. 5 mm – slice interval 1 -10 mm depending on anatomy – 50 -200 slices per study – noise in the presence of metal (blooming) • • All digital, printed on X-ray film Acquisition 1 sec/slice (spiral models) 15 mins for image reconstruction Costs about $250 -750 K, each study $500 CAS, Srping 2002 © L. Joskowicz 20

Magnetic Resonance Imaging • Similar principle and construction than CT machine, but works on magnetic properties of matter – magnetic fields of 0. 1 to 4 Teslas • • Similar image quality characteristics as CT Excellent resolution for soft tissue Costs $1 -2 M, each study $1, 000 Open MR: intraoperative device (only 15 to date) CAS, Srping 2002 © L. Joskowicz 21

Magnetic Resonance Images CAS, Srping 2002 © L. Joskowicz 22

Nuclear Medicine Imaging (NMI) • Same slices principle • Source of photons or positrons is injected in the body. Shortly after, radiation of metabolism is measured • Poor spatial resolution • Expensive machine AND installation ($4 -5 M) • Expensive and time-consuming • Provides functional info no other source does CAS, Srping 2002 © L. Joskowicz 23

Nuclear medicine images CAS, Srping 2002 © L. Joskowicz 24

Image Fusion: MRI and NMI MRI (anatomy) CAS, Srping 2002 © L. Joskowicz NMI (functional) 25

Video images from within the body • Used in laparoscopic and endoscopic surgery CAS, Srping 2002 © L. Joskowicz 26

Main medical imaging modalities X-ray Fluoro US US Video CT MRI NMR Open Film Digital (2 D) (2. 5 D) MR Pre/Intraop 2 D/2. 5 D Resolution Radiation Anatomy Procedure Establish a comparative table of modality properties CAS, Srping 2002 © L. Joskowicz 27

The imaging pipeline CAS, Srping 2002 © L. Joskowicz 28

2. Medical image visualization Enhance diagnosis by improving the visual interpretation of medical data • • 3 D visualization of complex structures image correlation and fusion quantitative measurements and comparisons visualization of medical and CAD data CAS, Srping 2002 © L. Joskowicz 29

Medical image visualization CAS, Srping 2002 © L. Joskowicz 30

Visualization: Technical needs • image enhancing and noise reduction • image interpolation: images from new viewpoints • 3 D visualization from 2. 5 D data – volume rendering: display voxels and opacity values – surface rendering: explicit reconstruction of surface • 3 D modeling from 2. 5 D data • 2 D and 3 D segmentation • 3 D+T visualization (beating heart) CAS, Srping 2002 © L. Joskowicz 31

Medical image visualization • Much activity! Radiologists are the experts • Commercial packages – 3 DVIEWNIX, ANALYZE, IMIPS • Main technical topics: – 3 D volume rendering techniques – 3 D image filtering and enhancement – surface construction algorithms: Marching cubes, etc. • Sources: chapters 3, 9, and 10 in textbook • Related fields: computer graphics, image processing CAS, Srping 2002 © L. Joskowicz 32

3. Segmentation and modeling Extract clinically useful information for a given task or procedure • Isolation of relevant anatomical structures based on pixel properties • Model creation for the next computational task – real-time interaction and visualization – simulation – registration, matching, – morphing CAS, Srping 2002 © L. Joskowicz 33

Segmentation and modeling CAS, Srping 2002 © L. Joskowicz 34

Segmentation and modeling: technical needs • Segmentation: – landmark feature detection – isosurface construction (Marching cubes) – contour extraction, region identification • Modeling: – points, anatomical landmarks, surface ridges – surfaces as polygon meshes, surface splines – model simplification methods (Alligator, Wrapper) CAS, Srping 2002 © L. Joskowicz 35

Segmentation and modeling • Medical images have very special needs! • Commercial packages – 3 DVIEWNIX, ANALYZE, IMIPS • Main technical topics: – Volumetric segmentation techniques for CT, MRI – 2 D and 3 D segmentation with deformable elements – surface and model simplification algorithms • Sources: chapters 4 and 8 in textbook • Related fields: image processing, computer vision CAS, Srping 2002 © L. Joskowicz 36

4. Virtual and augmented reality Use images to create or enhance a surgical situation • Create a virtual model for viewing during surgery • Project the model on the patient or integrate with surgeon’s view • Useful for intraoperative anatomy exploration and manipulation • Telesurgery systems CAS, Srping 2002 © L. Joskowicz 37

Virtual and augmented reality CAS, Srping 2002 © L. Joskowicz 38

Virtual and augmented reality • • Part manipulation, visual and sensory feedback Interaction devices: goggles, gloves, etc Only a handful of systems exist Main technical topics: – a couple of the working systems; simulators – telesurgery systems • Sources: chapters 14 and 15 in textbook • Related fields: computer graphics CAS, Srping 2002 © L. Joskowicz 39

5. Preoperative analysis and planning Use images and models to assist surgeons in planning a surgery and evaluate options • • • CAS, Srping 2002 Task and procedure dependent Spatial and volume measurements Stress and fracture analysis Implant and tool selection and positioning Surgical approach planning: bone rearrangement, angle evaluation, radiation dose planning, etc © L. Joskowicz 40

Preoperative analysis and planning CAS, Srping 2002 © L. Joskowicz 41

Preoperative analysis and planning • About a dozen planners exist for different procedures • Main technical topics: – planning systems for orthopaedics, neurosurgery – application of engineering analysis techniques: finiteelement methods, stress analysis, etc • Sources: chapters 11, 25, 33, 41, 52 --56 in textbook • Related fields: CAD, computational geometry, engineering analysis CAS, Srping 2002 © L. Joskowicz 42

6. Image and robot registration Establish a quantitative relation between different refererence frames • Define correspondance features – point-to-point, point-to-line, surface-to-surface • • Establish correspondances between features Establish a similarity measure Formulate and solve dissimilarity reduction problem Related tasks: image fusion, morphing, atlas matching CAS, Srping 2002 © L. Joskowicz 43

Multimodal registration problems • Great differences depending on – the type of data to be matched – the anatomy that is being imaged – the specific clinical requirements of procedures • Feature selection and extraction: • • CAS, Srping 2002 stereotactic frame, implanted fiducials, anatomical landmarks and surfaces, contours and surfaces in Manual vs. automatic feature selection, pairing Rigid vs. deformable registration Nearly similar vs. dissimilar images Noiseless vs. noisy images (outlier removal) © L. Joskowicz 44

Registration chain example Infrared tracker Tracker CT X-rays Instruments Patient 3 D surface model CAS, Srping 2002 © L. Joskowicz 45

Image and robot registration • • Rich topic of very great importance! Types of registration methods vary widely Main technical topics: rigid registration methods: three points and more – deformable registration: local and global methods – intensity-based registration • Sources: chapters 5 -7 in textbook, many papers Book on Medical Image Registration • Related fields: vision, robotics CAS, Srping 2002 © L. Joskowicz 46

7. Medical robotics devices Semi-active and active mechanical devices for improving surgical outcome • • • Task and procedure dependent Accurate, steady, and repeatable 3 D positioning Navigation and localization aids Cutting and milling, biopsies Key issues are: – kinematic design, trajectory planner – controller, safety provisions CAS, Srping 2002 © L. Joskowicz 47

Medical robotics devices CAS, Srping 2002 © L. Joskowicz 48

Medical robotics devices • Mostly passive and semiactive devices • Rich topic of very great importance! • Main technical topics: – compare features and functionalities of systems – discuss and compare design considerations – devices for specific surgeries laparoscopy) • Sources: chapters 16 -18, 22, 29, 34, 39, 45, 47, and 48 in textbook • Related fields: robotics, mechatronics CAS, Srping 2002 © L. Joskowicz 49

8. Real-Time tracking devices Hardware to follow in real time the precise position and orientation of anatomy and instruments during surgery • Ideally, an accurate Global Positioning System! • Current technologies offer only partial solution • Based on different principles – video: follow known objects – optical: follow light-emitting diodes – magnetic: measure the variation of – acoustic: works like a radar CAS, Srping 2002 © L. Joskowicz 50

Optical and video tracking devices camera instrument Passive markers Instrument has infrared LEDs attached to it CAS, Srping 2002 © L. Joskowicz Active markers 51

What kind of accuracy? CAS, Srping 2002 © L. Joskowicz 52

9. Safety, man-machine interfaces • Medical systems have very stringent safety requirements • Reported cases of radiation overdose due to faulty system design • Important issues in man-machine interfaces • Ideas for presentations – the radiotherapy accident – chapters 12 -15 and 19 in textbook CAS, Srping 2002 © L. Joskowicz 53

10. Systems integration • Complete systems that address specific clinical problems in domains • Use available technology to develop the system • The hard part: make it all work! • Main technical topics: – systems in orthopaedics, neurosurgery, etc • Sources: chapters in each section of • Related fields: all! CAS, Srping 2002 © L. Joskowicz 54