
3423e2411541cd6dca801dd50bdc9b95.ppt
- Количество слайдов: 63
03/02/10 Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem Acknowledgment: Many keypoint slides from Grauman&Leibe 2008 AAAI Tutorial
What is “object recognition”?
1. Identify a Specific Instance • General objects – Challenges: rotation, scale, occlusion, localization – Approaches • Geometric configurations of keypoints (Lowe 2004) – Works well for planar, textured objects
1. Identify a Specific Instance • Faces – Typical scenario: few examples per face, identify or verify test example – What’s hard: changes in expression, lighting, age, occlusion, viewpoint – Basic approaches (all nearest neighbor) 1. Project into a new subspace (or kernel space) (e. g. , “Eigenfaces”=PCA) 2. Measure face features 3. Make 3 d face model, compare shape+appearance (e. g. , AAM)
2. Detect Instance of a Category • Much harder than specific instance recognition • Challenges – Everything in instance recognition – Intraclass variation – Representation becomes crucial
2. Detect Instance of a Category • Template or sliding window • Works well when – Object fits well into rectangular window – Interior features are discriminative Schneiderman Kanade 2000
2. Detect Instance of a Category • Parts-based Fischler and Elschlager 1973 Felzenszwalb et al. 2008
3. Assign a label to a pixel or region • Stuff – Materials, object regions, textures, etc. – Approaches • Label patches + CRF • Segmentation + Label Regions
General Process of Object Recognition Specify Object Model Generate Hypotheses Score Hypotheses Resolution
General Process of Object Recognition Example: Template Matching Specify Object Model Intensity Template, at x-y Scanning window Generate Hypotheses Score Hypotheses Resolution Normalized X-Corr Threshold + Nonmax suppression
General Process of Object Recognition Example: Keypoint-based Instance Recognition Specify Object Model A 1 A 2 Generate Hypotheses B 3 Affine-variant point locations B 1 Affine Parameters Score Hypotheses # Inliers Resolution Choose hypothesis with max score above threshold B 2
General Process of Object Recognition Example: Keypoint-based Instance Recognition Specify Object Model A 1 A 2 Generate Hypotheses Score Hypotheses Resolution B 3 A 3 B 1 B 2 Today’s Class
Overview of Keypoint Matching 1. Find a set of distinctive keypoints B 3 A 1 A 2 2. Define a region around each keypoint A 3 B 2 B 1 3. Extract and normalize the region content 4. Compute a local descriptor from the normalized region K. Grauman, B. Leibe 5. Match local descriptors
Main challenges • Change in position and scale • Change in viewpoint • Occlusion • Articulation
Goals for Keypoints Detect points that are repeatable and distinctive
Key trade-offs B 3 A 1 A 2 A 3 B 1 B 2 Localization More Points More Repeatable Robust to occlusion Works with less texture Robust detection Precise localization Description More Robust More Selective Deal with expected variations Maximize correct matches Minimize wrong matches
Keypoint Localization • Goals: – Repeatable detection – Precise localization – Interesting content K. Grauman, B. Leibe
Choosing interest points • If you wanted to meet a friend would you say a) b) c) – “Let’s meet on campus. ” “Let’s meet on Green street. ” “Let’s meet at Green and Wright. ” Corner detection • Or if you were in a secluded area: a) b) c) – “Let’s meet in the Plains of Akbar. ” “Let’s meet on the side of Mt. Doom. ” “Let’s meet on top of Mt. Doom. ” Blob (valley/peak) detection
Choosing interest points • Corners – “Let’s meet at Green and Wright. ” • Peaks/Valleys – “Let’s meet on top of Mt. Doom. ”
Many Existing Detectors Available Hessian & Harris [Beaudet ‘ 78], [Harris ‘ 88] Laplacian, Do. G [Lindeberg ‘ 98], [Lowe 1999] Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘ 01] Harris-/Hessian-Affine[Mikolajczyk & Schmid ‘ 04] EBR and IBR [Tuytelaars & Van Gool ‘ 04] MSER [Matas ‘ 02] Salient Regions [Kadir & Brady ‘ 01] Others… K. Grauman, B. Leibe
Hessian Detector [Beaudet 78] • Hessian determinant Ixx Iyy Ixy Intuition: Search for strong derivatives in two orthogonal directions K. Grauman, B. Leibe
Hessian Detector [Beaudet 78] • Hessian determinant Ixx Iyy Ixy In Matlab: K. Grauman, B. Leibe
Hessian Detector – Responses [Beaudet 78] Effect: Responses mainly on corners and strongly textured areas.
Hessian Detector – Responses [Beaudet 78]
Harris Detector [Harris 88] • Second moment matrix (autocorrelation matrix) Intuition: Search for local neighborhoods where the image content has two main directions (eigenvectors). K. Grauman, B. Leibe
Harris Detector [Harris 88] • Second moment matrix (autocorrelation matrix) Ix Iy Ix 2 Iy 2 Ix Iy g(Ix 2) g(Iy 2) g(Ix. Iy) 1. Image derivatives 2. Square of derivatives 3. Gaussian filter g(s. I) 4. Cornerness function – both eigenvalues are strong 5. Non-maxima suppression g(Ix. Iy) 32 har
Harris Detector – Responses [Harris 88] Effect: A very precise corner detector.
Harris Detector – Responses [Harris 88]
So far: can localize in x-y, but not scale
Automatic Scale Selection How to find corresponding patch sizes? K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe
What Is A Useful Signature Function? • Laplacian-of-Gaussian = “blob” detector K. Grauman, B. Leibe
Laplacian-of-Gaussian (Lo. G) • Local maxima in scale space of Laplacian-of-s Gaussian 5 s 4 s 3 s 2 s K. Grauman, B. Leibe List of (x, y, s)
Results: Laplacian-of-Gaussian K. Grauman, B. Leibe
Difference-of-Gaussian (Do. G) • Difference of Gaussians as approximation of the Laplacian-of-Gaussian = - K. Grauman, B. Leibe
Do. G – Efficient Computation • Computation in Gaussian scale pyramid Sampling with step s 4 =2 s s s Original image s K. Grauman, B. Leibe
Results: Lowe’s Do. G K. Grauman, B. Leibe
Orientation Normalization • Compute orientation histogram [Lowe, SIFT, 1999] • Select dominant orientation • Normalize: rotate to fixed orientation 0 T. Tuytelaars, B. Leibe 2 p
Harris-Laplace [Mikolajczyk ‘ 01] 1. Initialization: Multiscale Harris corner detection s 4 s 3 s 2 s Computing Harris function Detecting local maxima
Harris-Laplace [Mikolajczyk ‘ 01] 1. Initialization: Multiscale Harris corner detection 2. Scale selection based on Laplacian (same procedure with Hessian-Laplace) Harris points Harris-Laplace points K. Grauman, B. Leibe
Maximally Stable Extremal Regions [Matas ‘ 02] • Based on Watershed segmentation algorithm • Select regions that stay stable over a large parameter range K. Grauman, B. Leibe
Example Results: MSER 54 K. Grauman, B. Leibe
Available at a web site near you… • For most local feature detectors, executables are available online: – http: //robots. ox. ac. uk/~vgg/research/affine – http: //www. cs. ubc. ca/~lowe/keypoints/ – http: //www. vision. ee. ethz. ch/~surf K. Grauman, B. Leibe
Local Descriptors • The ideal descriptor should be – Robust – Distinctive – Compact – Efficient • Most available descriptors focus on edge/gradient information – Capture texture information – Color rarely used K. Grauman, B. Leibe
Local Descriptors: SIFT Descriptor [Lowe, ICCV 1999] Histogram of oriented gradients • Captures important texture information • Robust to small translations / affine deformations K. Grauman, B. Leibe
Details of Lowe’s SIFT algorithm • Run Do. G detector – Find maxima in location/scale space – Remove edge points • Find all major orientations – Bin orientations into 36 bin histogram • Weight by gradient magnitude • Weight by distance to center (Gaussian-weighted mean) – Return orientations within 0. 8 of peak • Use parabola for better orientation fit • For each (x, y, scale, orientation), create descriptor: – – Sample 16 x 16 gradient mag. and rel. orientation Bin 4 x 4 samples into 4 x 4 histograms Threshold values to max of 0. 2, divide by L 2 norm Final descriptor: 4 x 4 x 8 normalized histograms Lowe IJCV 2004
Matching SIFT Descriptors • Nearest neighbor (Euclidean distance) • Threshold ratio of nearest to 2 nd nearest descriptor Lowe IJCV 2004
SIFT Repeatability Lowe IJCV 2004
SIFT Repeatability
SIFT Repeatability Lowe IJCV 2004
Local Descriptors: SURF • Fast approximation of SIFT idea Ø Ø Efficient computation by 2 D box filters & integral images 6 times faster than SIFT Equivalent quality for object identification • GPU implementation available Ø Ø [Bay, ECCV’ 06], [Cornelis, CVGPU’ 08] Feature extraction @ 200 Hz (detector + descriptor, 640× 480 img) http: //www. vision. ee. ethz. ch/~surf K. Grauman, B. Leibe
Local Descriptors: Shape Context Count the number of points inside each bin, e. g. : Count = 4. . . Count = 10 Log-polar binning: more precision for nearby points, more flexibility for farther points. Belongie & Malik, ICCV 2001 K. Grauman, B. Leibe
Local Descriptors: Geometric Blur Compute edges at four orientations Extract a patch in each channel ~ Apply spatially varying blur and sub-sample Example descriptor (Idealized signal) Berg & Malik, CVPR 2001 K. Grauman, B. Leibe
Choosing a detector • What do you want it for? – Precise localization in x-y: Harris – Good localization in scale: Difference of Gaussian – Flexible region shape: MSER • Best choice often application dependent – Harris-/Hessian-Laplace/Do. G work well for many natural categories – MSER works well for buildings and printed things • Why choose? – Get more points with more detectors • There have been extensive evaluations/comparisons – [Mikolajczyk et al. , IJCV’ 05, PAMI’ 05] – All detectors/descriptors shown here work well
Comparison of Keypoint Detectors Tuytelaars Mikolajczyk 2008
Choosing a descriptor • Again, need not stick to one • For object instance recognition or stitching, SIFT or variant is a good choice
Things to remember • Keypoint detection: repeatable and distinctive – Corners, blobs, stable regions – Harris, Do. G • Descriptors: robust and selective – spatial histograms of orientation – SIFT
Next time • Recognizing objects using keypoints