Chapter 9 Image Registration Chuan-Yu Chang (張傳育 )Ph. D. Dept. of Computer and Communication Engineering National Yunlin University of Science & Technology chuanyu@yuntech. edu. tw http: //mipl. yuntech. edu. tw Office: EB 212 Tel: 05 -5342601 Ext. 4337
Introduction n Different medical imaging modalities provide specific information about human physiology and physiological processes that are often complimentary in diagnosis. To understand the physiological processes better, images obtained from different modalities need to be registered. To study the variability of anatomical and function structures among the subjects, images from respective modalities can be registered to develop computerized atlases. q Structural Computerized Atlas (SCA) n q Represent the anatomical variations among subjects can be developed using registered image from the anatomical medical imaging modalities such as CT or MRI. Functional Computerized Atlas (FCA) n Represent the metabolic variations among subjects for a specific pathology or function can be developed using registered images from the functional medical imaging modalities such as f. MRI, SPECT or PET. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 2
A schematic diagram of multi-modality MR-PET image analysis using computerized atlases. Anatomical Reference (SCA) Reference Signatures MR Image (New Subject) MR-PET Registration Functional Reference (FCA) PET Image (New Subject) Analysis 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 3
Introduction n Image registration methods and algorithms provide transformation of a source image space to the target image space. q n The target image may be an image of the same or any other subject from any medical imaging modality. Registration methods q q q External markers and stereotactic frames based landmark registration. Rigid-body transformation based global registration. Image feature-based registration n n Boundary and surface matching based registration Image landmarks and features based registration 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 4
Image Registration Through Transformation A B f g 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 5
Rigid-Body Transformation q q q Rigid-Body Transformation is based on translation and rotation operations. Two images of equal dimensions are registered by applying a pixel-by-pixel transformation consistently throughout the image space. A rigid transformation based mapping of a point vector x to x’ is defined by where R is a rotation matrix and t is translation vector. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 6
Rigid-body Transform n Translation along x-axis by p n Translation along y-axis by q n Translation along z-axis by r Rotation by f Translation of z Translation of x Translation of y Rotation by q Rotation by w n The translation and rotation operations of a 3 -D rigid transformation. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 7
Rigid-body Transform n Rotation about x-axis by q n Rotation about y-axis by w n Rotation about z-axis by f Rotation by f Translation of z Translation of x Translation of y Rotation by q Rotation by w n The translation and rotation operations of a 3 -D rigid transformation. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 8
Rigid-body Transform q The rotation matrix R for the x-y-z rotational order of operation can be given as 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 9
Affine Transformation q q Affine transformation is a special case of rigidbody transformation that includes translation, rotation and scaling operations. If the two image volumes to be registered are not at the same scale, a scaling parameter in each dimension has to be added as where a, b, and c are the scaling parameters along x, y, and z directions. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 10
Affine Transformation q The affine transformation can be expressed as q where A is the Affine matrix that includes the translation, rotation and scaling transformation with nine parameters. The overall mapping can be expressed as 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 11
Principal Axes Registration q q q Principal Axes Registration can be used for global matching of two binary volumes such as segmented brain volumes from CT, MR or PET images. Let us represent a binary segmented B(x, y, z) as Let the centroid of the binary volume B(x, y, z) be represented by (xg, yg, zg)T 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 12
Principal Axes Registration q The principle axes of B(x, y, z) are the eigenvectors of the inertia matrix I: where q The method can resolve six degrees of freedom of an object including three rotations and three translations. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 13
Principal Axes Registration q Let us define a normalized eigenvector matrix E as q Let R=Ra. Rb. Rr represent the rotation matrix as where a, b, and r are the rotation angles with respect to the x, y, and z axes. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 14
Principal Axes Registration q By equating the normalized eigenvector matrix to the rotation matrix as it can be shown that 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 15
Principal Axes Registration q Given two volumes, V 1 and V 2, for registration, the PAR method provides the following operations: n n n Translate the centroid of V 1 to the origin. Rotate the principal axes of V 1 to coincide with the x, y and z axes. Rotate the x, y and z axes to coincide with the principal axes of V 2. Translate the origin to the centroid of V 2. The volume V 2 is scaled to match the volume V 1 using the scaling factor Fs. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 16
Principal axes Transformation Step 1: Define the volume 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 17
Principal axes Transformation n Step 3: Computing the principal axes: q The principal axes of V(x, y, z) are the eigenvectors of the inertia matrix I: 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 18
Principal axes Transformation The normalized eigenvector matrix E of I is then obtained with 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 19
Principal axes Transformation n Step 4: Computing the rotation matrix q The E is expanded to a product of rotation matrix by 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 20
Principal axes Transformation n Step 5: Computing the transform matrix q The registration of image 1 to image 2 can be obtained by a translation to the center of mass coordinate system followed by the transform matrix 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 21
q A 3 -D model of brain ventricles obtained from registering 22 MR brain images using the PAR method. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 22
q Rotated views of the 3 -D brain ventricle model shown in Figure 9. 3. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 23
Iterative Principal Axes Registration The Iterative Principal Axes Registration method can be used with partial volumes. n q q q For registering MR and PET brain images. The IPAR algorithm allows registration of two 3 D image data sets in which one of the data set does not cover the entire volume but has the subvolume contained in the other data set. Let V 1 and V 2 represent two volumes to be registered, the IPAR method can be implemented using the following steps: 1. Find the full dynamic range of PET data and select a threshold T, which is about 20% of the maximum graylevel value. Extract binary brain regions using a region growing method on the thresholded PET slice data. 2. Threshold and extract binary brain regions from the MR data using a region growing method. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 24
Iterative Principal Axes Registration 3. 4. 5. 6. Stack 2 -D binary segmented MR slices and interpolate as necessary to obtain cubic voxel dimensions using a shape-based interpolation algorithm. (3 -D binary MR data) Stack 2 -D binary segmented PET slices and interpolate as necessary to obtain cubic voxel dimension to match the voxel dimension of brain MR data using a shape-based interpolation algorithm. . (3 -D binary PET data) Define a Field of View box, FOV(0) as a parallelepiped from the slices of the interpolated binary PET data to cover the PET brain volume Compute the centroid and principal axes of the binary PET brain volume. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 25
Iterative Principal Axes Registration 7. 8. 9. IPAR algorithm Interpolate the gray-level PET data to match the resolution of MR data to prepare the PET data for registration with MR data. Transform the gray-level PET data into the space of the MR slices using the last set of MR and PET centroids and principal axes. Extract the slices from the transformed gray-level PET data that match the graylevel MR image. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 26
Iterative Principal Axes Registration The IPAR algorithm n For i =n to 0 do q n n n Compute the centroid and principal axes of the current binary MR brain volume. Transform the augmented FOV (i) box according to the space of the MR slices The PET data are registered with the MR data by performing the required translations and rotations 1. 2. 3. 4. n Translate the centroid of the binary PET data to origin. Rotate the principal axes of the binary PET data to coincide with the x, y and z axes. Rotate the x, y and z axes to coincide with the MR principal axes. Translate the origin to the centroid of the binary MR data. Remove all voxels of the binary MR brain which lie outside the transformed FOV(i) box. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 27
Iteration 1 n Iteration 2 Three successive iterations of the IPAR algorithms for registration of vol 1 and vol 2: The results of the first iteration (a), the second iteration (b) and the final iteration (c). Vol 1 represents the MR data while the PET image with limited filed of view (FOV) is represented by vol 2. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 28
Iteration 3 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 29
n Sequential slices of MR (middle rows) and PET (bottom rows) and the registered MR-PET brain images (top row) of the corresponding slices using the IPAR method. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 30
n Sequential slices of MR (middle rows) and PET (bottom rows) and the registered MR-PET brain images (top row) of the corresponding slices using the IPAR method. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 31
n Sequential slices of MR (middle rows) and PET (bottom rows) and the registered MR-PET brain images (top row) of the corresponding slices using the IPAR method. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 32
Image Landmarks and Features based Registration q Once the corresponding landmarks or features are identified from in source and target image spaces, a customized transformation can be computed for registering the source image into the target image space. n n Relationships of corresponding points Relationships of corresponding feature such as surface 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 33
Similarity Transformation for Point. Based Registration q q Assume that x and y are the corresponding points in the source and target image spaces belonging to the source X and target Y images. A non-rigid transformation T(x) for registering the source image into the target image space can be defined by a combination of rotation, translation and scaling operations to provide x’ from x as such that the registration error E is minimized as where r, s and t represent the rotation, scaling and translation operations. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 34
Similarity Transformation for Point. Based Registration q A transformation should be obtained with r, s and t values to minimize the error function as where wis are the weighting factors representing the confidence in the specific landmark (point) or feature correspondence and N is the total number of landmarks. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 35
Similarity Transformation for Point-Based Registration To register the source image into the target image space q 1. 2. Set s=1 Find r through the following steps q Compute the weighted centroid of the body representing the set of landmarks in each spaces as q Compute the distance of each landmark from the centroid as 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 36
Similarity Transformation for Point-Based Registration n Compute the weighted co-variance matrix as with a singular value decomposition as where Ut. U=Vt. V=I and n Compute 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 37
Similarity Transformation for Point-Based Registration 3. Compute the scaling factor 4. Compute the translation factor 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 38
Point-Based Registration R is a 3 x 3 rotation matrix, t is a 3 x 1 translation vector, p is a 3 x 1 position vector. <=Orthogonal Procrustes 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 39
Surface-Based Registration n Outlining contours on the serial slices of each scan. Head is a stack of disks or “prisms”, each of which has cross section determined by one of the contour. Hat is represented as a set of independent points. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 40
Weighted Feature Based Registration q q q Different optimization functions can be designed to improve the computation of parameters of transformation for registration of the source image into the target image space. A disparity function can be designed as where {Xi} for I = 1, 2, 3, …, N represents a set of corresponding data shapes in x and y spaces. The transformation T must minimize the disparity function register the source image into the target space utilizing the correspondence of geometrical features. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 41
Weighted Feature Based Registration q q q Determine the parameters for a rigid or non-rigid body transformation T. Initialize the transformation optimization loop for k=1 as For each shape Xi in the source space, find the closest points in the corresponding shape in the target space Yi as where Ci is the corresponding function. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 42
Weighted Feature Based Registration q q q Compute the transformation between {xij(0)} and {yij(k)} with the weights {wij}. Use the transformation parameters for registration of the corresponding points as Compute the disparity measure difference d(T(k))d(T(k+1)), if the convergence criterion is met, stop; otherwise increment k and go to step 3 for next iteration. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 43
Point-Based Registration n Determinate the coordinates of corresponding points in different images, and the estimation of the geometrical transformation using these corresponding point. Intrinsic points: anatomic landmark Extrinsic points: artificially applied markers 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 44
Surface-Based Registration The general approach is to search iteratively for the rigid-body transformation T that minimizes the cost function: is a point on the surface X C is a correspondence function. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 45
Surface-Based Registration(2) n n Coarse registration: Principal axes Transformation Fine registration: surface fitting A rigid body is determined by the position of its center of mass and its orientation with respect to its center of mass (principal axes) Compute the centroid and the three principal axes for a 3 -D volume data 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 46
Surface fitting based on (E-Distance Transformation) n n The DT is performed on the 1 st surface images (base image) base image 2 nd surface image (match image) q to determine the registration parameters 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 47
Surface fitting based on (E-Distance Transformation) DQ is the distance map of the base image Q pn is the nth point of match image P 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 48
ICP Registration algorithm n Iterative Closest Point, ICP 1. assigning one shape to be the “data” shape 2. assigning other shape to be the “model” shape 3. The “data” shape is decomposed into a point set 4. The “data” shape is registered to the “model” shape by iteratively finding “model” points closest to the “data” primitives. q ICP registration method defines the “corresponding” point yj to be the “closest” point on the surface 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 49
Volume-Based Registration n n 1. Roger P. Woods, Simon R. Cherry, and John C. Mazziotta, ”Rapid automated algorithm for aligning and reslicing PET images”, Journal of Computer Assisted Tomography, 16(4): 620 -633, 1992. 2. Roger P. Woods, John C. Mazziotta, and Simon R. Cherry, ”MRI-PET registration with automated algorithm”, Journal of Computer Assisted Tomography, 17(4): 536 -546, 1993. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 50
Basic Assumption q If two image sets are accurately aligned, then the value of any voxel in one image set is related to the value of the corresponding voxel in the other image set by a single multiplicative factor, R. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 51
Basic Assumption 令 ri=ai/bi where ai and bi is the value of voxel i in reference study and the corresponding voxel in reslice study. r: standard deviation of ri over all voxels within the brain. rmean: mean value of ri over all voxels within the brain. Objective: Minimize the r / rmean 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 52
醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 53
Image Alignment Algorithm Step 1: 使用者指定 Reference study 與 Reslice study. Step 2: 使 用 者 初 步 評 估 六 個 reslice parameters, (x-, y, z-的旋轉與平移 ) Step 3: Linear interpolation in z-axis, 產生 3 D reference volume. Step 4: 利 用 Step 2的 reslice parameters 以 Trilinear interpolation 法 , 產生 3 D reslice volume. Step 5: 計算 ratio volumn ri=ai/bi Step 6: 利 用 thresholding法 , 只 留 下 腦 部 區 域. 並 將 其 他 地 區 的 ratio volumn設 為 0. (對 reference volume) 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 54
Image Alignment Algorithm Step 7: 計算平均 ratio volumn rmean, 及正規化的標準差 r / rmean. (此比值越小 , 代表越 uniform) Step 8: 調整 reslice parameters, 求出使 brain/ rbrain最小 化 的 reslice parameters參數值. Disadvantage: 1. 無法進行不同模組影像間的對準 2. 計算複雜度很高 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 55
MRI-PET Registration n Two modifications: q q q Edit the MR images to exclude nonbrain structures prior to registration. Partitions the MR image into 256 separate components based on the value of the MR pixels. Seeks to maximize the uniformity of the PET pixel values within each of these partition. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 56
MRI-PET Registration MRI為reference PET為reslice Step 1: 移除 MRI中非腦部區域. (人 ) Step 2: 使用者評估六個 reslice parameters, (x-, y-, z-的旋轉與平移 ) Step 3: Construct the 3 D volume of MRI and PET Step 4: 計算 MRI與 PET中相對應 voxels的平均值 a’j及標準差 j Step 5: 計算 Step 6: 計算 Step 7: 調整 reslice parameters使 ‘’最小化 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 57
MRI-PET Registration n Disadvantage q q Segmentation是一件困難的事 無法適用於所有類型的醫學影像 , (打藥前後 ) 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 58
不同對準法的比較 n 2 -D Registration q Point-Based Registration n q 缺 點 : 必 須 找 出 欲 對 準 影 像 的 特 徵 點 包 括 Intrinsic points( anatomic landmark) 及 Extrinsic points( artificially applied markers) Lvv n 優點 : q n 不需對影像進行特徵擷取的動作 . 缺點 : q 由 於 不 同 模 組 影 像 特 性 不 同 , 造 成 ridge image會 存 在 許 多 差 異. 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 59
不同對準法的比較 n 3 -D Registration q Surface-Based Registration n 缺點 : q q q 1. 必須建立 3 -D volume 2. 必 須 找 出 每 張 影 像 的 輪 廓 , 以 輪 廓 來 進 行 , 因 此 只 適 用 於 相 同 modality的影像. Volume-Based Registration (R. P. Woods ’ 92, ‘ 93) n 缺點 : q q q 1. 無法進行不同模組影像間的對準 2. 計算複雜度很高 3. 不 同 模 組 影 像 的 對 準 必 須 事 先 segmentation, 可 是 segmentation是一件困難的事 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 60
不同對準法的比較 q Iterative Closest Point(ICP) n 缺點 : 1. 必 須 事 先 求 取 影 像 的 幾 何 特 徵. (point sets, line segment sets, triangle sets, surfaces, …) q 2. q 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 61
Block diagram for the MR image registration procedure 醫學影像處理實驗室 (Medical Image Processing Lab. ) Chuan-Yu Chang Ph. D. 62
Скачать презентацию Chapter 9 Image Registration Chuan-Yu Chang 張傳育 Ph
558e43b59dff899a9bfe9ec38a8a8534.ppt