Segmentation of Polarimetric SAR Data Using Spectral Graph

Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Kaan Ersahin*, Ian Cumming and Rabab K. Ward Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada

OUTLINE n Motivation n Using Ideas from HVS n Spectral Graph Partitioning (SGP) q Utilizing patch-based similarity in SGP q Utilizing contour information in SGP n Proposed Scheme n Results n Summary n Future Work Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 2

Motivation n Manual segmentation of SAR data is a common practice q n Human experts are often good at visual interpretation Operational use of polarimetric spaceborne systems means: q q n Daily acquisitions more data to analyze Wider spectrum of users with limited or no expertise in SAR Polarimetry Automated analysis procedures are needed q n To develop better decision making tools that require less analyst (human) interaction Analysis typically involves: Segmentation q e. g. , drawing boundaries between agricultural fields, water - ice separation, etc. q Automated segmentation task is very challenging n q Edge detection followed by linking or region merging methods often do not perform well Human vision system (HVS) can perform this task easily n Identify lines, contours, patterns and regions and make decisions based on global information Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 3

Importance of using global view Global view Local view © CSA 2004 Convair-580, C-band, color composite Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 4

Motivation – Developing a better method n A number of useful analysis techniques have been developed q q Eigenvalue decomposition H / A / α-angle (Cloude - Pottier) q Target decomposition based on physical models (Freeman - Durden) q n ML classifier based on Wishart distribution (Lee et. al ) … their combinations and variants These are based on polarimetric attributes of pixels (or averages in a neighborhood) q n Not able to capture the information that human observer can pick up Visual aspect of image data can be used to enhance automated segmentation results q Study how humans handle this task q Use the ideas that have led to the state-of-the-art technique in computer vision Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 5

Using Ideas from HVS n In computer vision problems (e. g. , segmentation, object detection) q n What does an image mean for humans? q n The ultimate goal To reach the performance level of an human expert More than the collection of pixels, represents a meaningful organization of objects or patterns In late 1930 s, Gestalt Psychologists 1 studied this phenomenon: perceptual organization q Several cues (i. e. , factors that contribute to this process) were reported: Similarity Proximity (e. g. , brightness, color, shape) 1 Closure (geometric) XXXX n Continuity OXXX XOXX XXOX XXXO XXXXXX In computer vision, a promising technique that can utilize these ideas has emerged: Spectral Graph Partitioning Gestalt: a configuration or pattern of elements so unified as a whole that its properties cannot be derived from a simple summation of its parts. 6

Spectral Graph Partitioning central grouping A pair-wise grouping technique: an alternative to (SGP) n q No assumption on the statistical distribution of the data (e. g. , Gaussian) q Avoids the restriction that all the points must be similar to a prototype (i. e. , class mean) n Enables combination of multiple cues (e. g. , different types of features and data sets) n Offers flexibility in the definition of affinity functions (i. e. , measure of similarity) G n W G = { V , E } is an undirected graph q V nodes (data points or pixels) q E edges (connections between node pairs) q ( i , j ) weights (similarity between node i and node j ) q W similarity matrix ; its entries are the weights: ( i , j ) Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 7

Spectral Graph Partitioning n To divide the graph into two partitions, intuitively: q similarity between the resulting partitions or q cost of removing all the connections between the candidate partitions (i. e. , cut) should be minimized n A better way: Minimize the Normalized Cut n Shi and Malik (2000) showed that solving the eigenvalue problem for the Normalized Graph Laplacian: provides a reasonable solution. n Yu and Shi (2003) showed that eigenvectors completely characterize all optimal solutions q q q Space of global optima can be navigated via orthogonal transforms. Iteratively solve for a discrete solution that is closest to the continuous global optimum using an alternating optimization procedure Their method is called Multiclass Spectral Clustering (MSC). Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 8

Utilizing Patch-based Similarity in SGP Pol. SAR Data n Multi-looking We have used SGP for classification based on patchbased similarity (IGARSS 2006) q Speckle Reduction n Form affinity matrix (W) n q Spectral Graph Partitioning Spectral Clustering algorithm is modified to account for the unique properties of SAR data q Instead of pixel intensities, the histograms calculated within an edge-aligned window mask are used as attributes. Similarity is measured using the 2 – distance Form an affinity matrix to account for spatial proximity Patch-based similarity cues from multiple channels and proximity are combined in an overall affinity matrix (W) Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 9

Utilizing Contour Information in SGP n In IGARSS 2007 we used SGP for segmentation based on contour information. The motivation was: q Region-based techniques perform either: n n Optimization of a global objective function n q Sequential merging of segments based on an appropriate measure (e. g. , likelihood ratio test) Drawback: contour information – a powerful cue for HVS – is not utilized. Contour-based techniques often start with edge detection, followed by a linking process. n q Drawback: Only local information is used; decisions on segment boundaries are made prematurely Leung and Malik addressed this issue by collecting contour information locally (i. e. , through orientation energy (OE), but making the decision globally. Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 10

Utilizing Contour Information in SGP Orientation Energy at orientation angle of 0 Rotated copies of filters will pick up edge contrast at different orientations: Orientation energy of a pixel located at (x, y) Useful properties: q q and form a quadrature pair. Filters are elongated, information is integrated along the edge Extended contours will stand out as opposed to short ones Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 11

Utilizing Contour Information in SGP Dissimilarity of two pixels n Based on the presence of an extended contour, pixel pairs can be assigned to same or different partitions q q n OE is strong along l 2 s 1 and s 3 are in different partitions. OE is weak along l 1 s 1 and s 2 are in the same partition. Pairwise affinity matrix is formed using Eq. 10: Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 12

Utilizing Contour Information in SGP Pol. SAR Data Multi-looking Form affinity matrix (W) Spectral Graph Partitioning n Perform multi-looking on SLC data set n Form affinity matrix for each channel based on OE n To account for proximity in the image plane calculate affinities only within a neighborhood. n Perform the steps of Multiclass Spectral Clustering (MSC) algorithm by Yu and Shi. Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning 13

Proposed Scheme Pol. SAR Data n Perform multi-looking on SLC data set n Multi-looking Form affinity matrix W for each data channel Proximity q Contour Information q To account for proximity in the image plane calculate affinities only within a neighborhood. Contour information is measured using Orientation Energy (OE) n Perform the Spectral Graph Partitioning (SGP) using the Multiclass Spectral Clustering (MSC) algorithm. n Form affinity matrix W and perform SGP Patch-based similarity q SGP q Similarity is defined between segments obtained from the previous step. ( 2 – distance between the histograms is used) Only consider adjacent segments Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 14

Data Set: Westham Island, B. C. n Data Acquisition: q n Convair-580, C - band, Sept. 2004 For the regions # 1 and # 2 the reference segmentation was formed by: q Inspection of the field boundaries and crops on the day of the acquisition q Visual interpretation of the image data q Manual Segmentation © CSA 2004 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 15

Data Set: Westham Island, B. C. n For region #3 a classification map was formed using: q q q GPS measurements at the field boundaries Inspection of the crops in each field on the day of the acquisition Visual interpretation of the image data © CSA 2004 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 16

Data Set: Westham Island, B. C. Potatoes Hay Barley -1 Pumpkin Bare Soil Barley – 2 Turnip Strawberry Segmentation of Polarimetric SAR Data Using Contour Information via Spectral Graph Partitioning 17

Results – Region # 1 Wishart n 6 fields n Wishart result contains isolated pixels n Proposed Method: q More homogenous q Visually agrees with reference segmentation Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 18

Results – Region # 2 Wishart n 8 fields n Wishart result contains isolated pixels n Proposed Method: q q More homogenous Visually agrees with reference segmentation Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 19

Results – Region # 3 n 13 different fields Pumpkin n Grass Problems: q q q Adjacent fields with same crop type Concave regions (Similarity calculation using OE suggests there should be two partitions Non-adjacent fields with same crop type. (To be solved at the level of classification) Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 20

Summary n A new technique for segmentation of polarimetric SAR data is proposed q Motivated by the visual information content that humans utilize q Is based on SGP which was shown to perform well on computer vision problems n A pair-wise grouping technique instead of central grouping. q q n Contour cue and Proximity is used for initial partitioning Patch-based similarity is used later to merge adjacent partitions Preliminary results are given on image subsets of Convair-580 data (C-band) q q n Perceptually plausible results: more homogenous, agree with the reference (i. e. , manual) segmentation Resulting classification is better than Wishart This scheme is flexible to allow further improvement using additional information Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 21

Future Work n Utilize the complete polarimetric information using pairwise similarity of the coherency matrices. n Include additional information (e. g. , scattering mechanisms) n Optimize the cue combination scheme n Compare with techniques other than Wishart n Validate methodology for q Different datasets (CV-580) q RADARSAT-2 Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues 22

Segmentation of Polarimetric SAR Data Using Spectral Graph Partitioning: Utilizing Multiple Cues Kaan Ersahin*, Ian Cumming and Rabab K. Ward Dept. of Electrical and Computer Engineering University of British Columbia Vancouver, Canada