Скачать презентацию The Uses of Object Shape from Images in Скачать презентацию The Uses of Object Shape from Images in

d199feb5c77224ea2a63d413de5998e2.ppt

  • Количество слайдов: 51

The Uses of Object Shape from Images in Medicine Stephen M. Pizer Kenan Professor The Uses of Object Shape from Images in Medicine Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina Credits: Many on MIDAG, especially Daniel Fritsch, Guido Gerig, Edward Chaney, Elizabeth Bullitt, Stephen Aylward, George Stetten, Gregg Tracton, Tom Fletcher, Andrew Thall, Paul Yushkevich, Nikki Levine, Greg Clary, David Chen [email protected]

Object Representation in Medical Image Analysis ä Extract an object from image(s) [segmentation] ä Object Representation in Medical Image Analysis ä Extract an object from image(s) [segmentation] ä Radiotherapy Tumor; plan to hit it ä Radiosensitive normal anatomy; plan to miss it ä PD MRA T 2 T 1 Contrast ä Surgery Plan to remove it ä Plan to miss it ä During surgery, view where it is & effect of treatment ä ä Radiology ä View it to judge its pathology [email protected]

Image Guided Planning of Radiotherapy ä Planning in 3 D ä Extracting normal anatomy Image Guided Planning of Radiotherapy ä Planning in 3 D ä Extracting normal anatomy ä Extracting tumor ä Planning beam poses [email protected]

Object Representation in Medical Image Analysis ä Registration (find geometric transformation that brings two Object Representation in Medical Image Analysis ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3 D/3 D) for planning ä Verify patient placement (3 D/2 D) ä Surgery ä Fuse multimodality images (3 D/3 D or 2 D) for planning ä Fuse preoperative (3 D) & intraoperative (2 D) images ä Radiology ä Fuse multimodality images (3 D/3 D) for diagnosis [email protected]

Object Representation in Medical Image Analysis ä Shape & Volume Measurement ä Make physical Object Representation in Medical Image Analysis ä Shape & Volume Measurement ä Make physical measurement ä Radiotherapy ä Measure effect of therapy on tumor ä Radiology, ä Neurosciences Use measurement in science of object development ä Find how probable an object is ä Radiology, Neurosciences Use measurement as quantitative input to diagnosis ä Use measurement in science of object development ä ä Use as prior in object extraction ä E. g. , extract the kidney shaped object [email protected]

Object Shape & Volume Measurement: Neurofibromatosis (Gerig, Greenwood) Infant Ventricle from 3 D U/S Object Shape & Volume Measurement: Neurofibromatosis (Gerig, Greenwood) Infant Ventricle from 3 D U/S (Gerig, Gilmore) [email protected]

Object Extraction (Segmentation) ä Approach 1: preanalyze, then fit to model ä Neurosurgery (MR Object Extraction (Segmentation) ä Approach 1: preanalyze, then fit to model ä Neurosurgery (MR Angiogram), Radiology (CT) ä Vessels, ribs, bronchi, bowel via tube skeletons ä Cardiology (3 D Ultrasound) Geometry via clouds of medial atoms ä Fit appropriately labeled clouds to 3 D LV model ä ä Cardiac Nuclear Medicine (2 D Gated Blood Pool Cine) Extract LV, with previous frame providing model ä Extraction via deformable m-rep model ä Shape from extracted LV; analyze shape series ä ä Surgery, Radiation Oncology (Multimodality MRI) ä Extract tumor, using local shape characteristics [email protected]

Extracting Trees of Vessels via Skeletons (Aylward, Bullitt) MIDAG@UNC Extracting Trees of Vessels via Skeletons (Aylward, Bullitt) [email protected]UNC

Presenting Ribs via Tube Skeletons (Aylward) MIDAG@UNC Presenting Ribs via Tube Skeletons (Aylward) [email protected]

Presenting Bronchi and Lung Vessels via Tube Skeletons (Aylward) MIDAG@UNC Presenting Bronchi and Lung Vessels via Tube Skeletons (Aylward) [email protected]

Presenting Small Bowel via Tube Skeletons (Aylward) MIDAG@UNC Presenting Small Bowel via Tube Skeletons (Aylward) [email protected]

Presenting Blood Vessels Supplying a Tumor for Embolization (Bullitt) Full tree, 2 D Subtree, Presenting Blood Vessels Supplying a Tumor for Embolization (Bullitt) Full tree, 2 D Subtree, 2 D 3 D, from 2 poses [email protected]

Heart Model (G. Stetten) MIDAG@UNC Heart Model (G. Stetten) [email protected]

Statistical Analysis of Medial Atom Clouds (G. Stetten) MIDAG@UNC Statistical Analysis of Medial Atom Clouds (G. Stetten) [email protected]

LV Tube Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder MIDAG@UNC LV Tube Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder [email protected]

Mitral Valve Slab Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder Mitral Valve Slab Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder [email protected]

Automatic LV Extraction via Mitral Valve/LV Tube Axis (G. Stetten) MIDAG@UNC Automatic LV Extraction via Mitral Valve/LV Tube Axis (G. Stetten) [email protected]

Gated Blood Pool Cardiac LV Cine Shape Analysis (G. Clary) Example sequence 4 -sided Gated Blood Pool Cardiac LV Cine Shape Analysis (G. Clary) Example sequence 4 -sided medial elliptical analysis [email protected]

Object Extraction (Segmentation) ä Approach 2: deform model to optimize reward for image match Object Extraction (Segmentation) ä Approach 2: deform model to optimize reward for image match + reward for shape normality ä Radiation Oncology (CT or MRI) ä Abdominal, pelvic organs ä Deform m-reps model ä Neurosciences (MRI or 3 D Ultrasound) ä Internal brain structures ä Spherical harmonics boundary model ä Deformable m-reps model ä Neurosurgery (CT) ä Vertebrae [email protected]

M- Reps for Medical Image Object Extraction and Presentation (Chen, Thall) MIDAG@UNC M- Reps for Medical Image Object Extraction and Presentation (Chen, Thall) [email protected]

Displacements from Figurally Implied Boundary implied by figural model Boundary after displacements MIDAG@UNC Displacements from Figurally Implied Boundary implied by figural model Boundary after displacements [email protected]

Vertebral M-reps Model MIDAG@UNC Vertebral M-reps Model [email protected]

Vertebral M-reps Model: Spinous Process Figure MIDAG@UNC Vertebral M-reps Model: Spinous Process Figure [email protected]

Cerebral Ventricle M-reps Model MIDAG@UNC Cerebral Ventricle M-reps Model [email protected]

Extraction with Object Shape as a Prior Brain structures (Gerig) MIDAG@UNC Extraction with Object Shape as a Prior Brain structures (Gerig) [email protected]

Registration ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy Registration ä Registration (find geometric transformation that brings two images into alignment) ä Radiotherapy ä Fuse multimodality images (3 D/3 D) for planning ä Verify patient placement (3 D/2 D) ä Surgery ä Fuse multimodality images (3 D/3 D or 2 D) for planning ä Fuse preoperative (3 D) & intraoperative (2 D) images ä Radiology ä Fuse multimodality images (3 D/3 D) for diagnosis [email protected]

Image Guided Delivery of Radiotherapy ä Patient placement ä Verification of plan via portal Image Guided Delivery of Radiotherapy ä Patient placement ä Verification of plan via portal image ä Calculation of new treatment pose [email protected]

Finding Treatment Pose from Portal Radiograph and Planning DRR MIDAG@UNC Finding Treatment Pose from Portal Radiograph and Planning DRR [email protected]

Medial Net Shape Models Medial nets, positions only MIDAG@UNC Medial Net Shape Models Medial nets, positions only [email protected]

Image Match Measurment of M-rep MIDAG@UNC Image Match Measurment of M-rep [email protected]

Registration Using Lung Medial Object Model : Reference Radiograph (Levine) Medial nets, positions only Registration Using Lung Medial Object Model : Reference Radiograph (Levine) Medial nets, positions only [email protected]

Radiograph/Portal Image Registration (Levine) Intensity Matching Relative to Medial Model Medial net MIDAG@UNC Radiograph/Portal Image Registration (Levine) Intensity Matching Relative to Medial Model Medial net [email protected]

Shape & Volume Measurement ä Find how probable an object is ä Training images; Shape & Volume Measurement ä Find how probable an object is ä Training images; Principal components ä Global vs. global and local ä Correspondence Hippocampi (Gerig) [email protected]

Modes of Global Deformation Training set: Mode 1: x = xmean + b 1 Modes of Global Deformation Training set: Mode 1: x = xmean + b 1 p 1 Mode 2: x = xmean + b 2 p 2 Mode 3: x = xmean + b 3 p 3 [email protected]

Shape & Volume Measurement ä Shape Measurement ä Modes of shape variation across patients Shape & Volume Measurement ä Shape Measurement ä Modes of shape variation across patients ä Measurement = amount of each mode Hippocampi (Gerig) [email protected]

Multiscale Medial Model ä From larger scale medial net, interpolate smaller scale medial net Multiscale Medial Model ä From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b. [email protected]

Summary: What shape representation is for in medicine ä Analysis from images ä Extract Summary: What shape representation is for in medicine ä Analysis from images ä Extract the “anatomic object”-shaped object ä Register based on the objects ä Diagnose based on shape and volume ä Medical science via shape ä Shape and biology ä Shape-based diagnostic approaches ä Shape-based therapy planning and delivery approaches [email protected]

Shape Sciences ä Medicine ä Biology ä Geometry ä Statistics ä Image Analysis ä Shape Sciences ä Medicine ä Biology ä Geometry ä Statistics ä Image Analysis ä Computer Graphics [email protected]

Options for Primitives ä Space: xi for grid elements ä Landmarks: xi described by Options for Primitives ä Space: xi for grid elements ä Landmarks: xi described by local geometry ä Boundary: (xi , normali) spaced along boundary ä Figural: nets of diatoms sampling figures [email protected]

Figural Models ä Figures: successive medial involution ä ä ä Main figure Protrusions Indentations Figural Models ä Figures: successive medial involution ä ä ä Main figure Protrusions Indentations Separate figures Hierarchy of figures ä ä ä Relative position Relative width Relative orientation [email protected]

Figural Models with Boundary Deviations ä Hypothesis ä At a global level, a figural Figural Models with Boundary Deviations ä Hypothesis ä At a global level, a figural model is the most intuitive ä At a local level, boundary deviations are most intuitive [email protected]

Medial Atoms ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom Medial Atoms ä Imply boundary segments with tolerance ä Similarity transform equivariant ä Zoom invariance implies width-proportionality of ä tolerance of implied boundary ä boundary curvature distribution ä spacing along net ä interrogation aperture for image [email protected]

Need for Special End Primitives ä Represent ä non-blobby objects ä angulated edges, corners, Need for Special End Primitives ä Represent ä non-blobby objects ä angulated edges, corners, creases ä still allow rounded edges , corners, creases ä allow bent edges ä But ä Avoid infinitely fine medial sampling ä Maintain tangency, symmetry principles [email protected]

Coarse-to-fine representation ä For each of three levels ä Figural hierarchy ä For each Coarse-to-fine representation ä For each of three levels ä Figural hierarchy ä For each figure, net chain, successively smaller tolerance ä For each net tile, boundary displacement chain [email protected]

Multiscale Medial Model ä From larger scale medial net ä Coarsely sampled ä Smooother Multiscale Medial Model ä From larger scale medial net ä Coarsely sampled ä Smooother figurally implied boundary ä Larger tolerance ä Interpolate smaller scale medial net ä Finer sampled ä More detail in figurally implied boundary ä Smaller tolerance ä Represent medial displacements [email protected]

Multiscale Medial/Boundary Model ä From medial net ä Coarsely sampled, smoother implied boundary ä Multiscale Medial/Boundary Model ä From medial net ä Coarsely sampled, smoother implied boundary ä Larger tolerance ä Represent boundary displacements along implied normals ä Finer sampled, more detail in boundary ä Smaller tolerance [email protected]

Shape Repres’n in Image Analysis ä Segmentation äFind the most probable deformed mean model, Shape Repres’n in Image Analysis ä Segmentation äFind the most probable deformed mean model, given the image äProbability involves ä Probability of the deformed model ä Probability of the image, given the deformed model [email protected]

Medialness: medial strength of a medial primitive in an image ä Probability of image Medialness: medial strength of a medial primitive in an image ä Probability of image | deformed model ä Sum of boundariness values ä at implied boundary positions ä in implied normal directions ä with apertures proportional to tolerance ä Boundariness value ä Intensity profile distance from mean (at scale) [email protected]

Shape Rep’n in Image Analysis ä Segmentation ä Find the most probable deformed mean Shape Rep’n in Image Analysis ä Segmentation ä Find the most probable deformed mean model, given the image ä Registration ä Find the most probable deformation, given the image ä Shape Measurement ä Find how probable a deformed model is [email protected]

Object Shape Representations for Medicine to Manufacturing ä Figural models, at successive levels of Object Shape Representations for Medicine to Manufacturing ä Figural models, at successive levels of tolerance ä Boundary displacements ä Work in progress ä Segmentation and registration tools ä Statistical analysis of object populations ä CAD tools, incl. direct rendering ä… [email protected]