Скачать презентацию 1 Face Recognition in the Infrared Spectrum Prof Скачать презентацию 1 Face Recognition in the Infrared Spectrum Prof

9c31cee5a3f0a57edfcf3b4e4e2ed465.ppt

  • Количество слайдов: 40

1 Face Recognition in the Infrared Spectrum Prof. Ioannis Pavlidis COSC 6397 – Lecture 1 Face Recognition in the Infrared Spectrum Prof. Ioannis Pavlidis COSC 6397 – Lecture 10 U of H

Primary Applications 2 • Biometric Identification – Passwords/PINs. – Tokens (like ID cards). – Primary Applications 2 • Biometric Identification – Passwords/PINs. – Tokens (like ID cards). – You can be your own password. • Surveillance – Off-the-shelf facial recognition system that identifies humans as they pass through a camera’s field of view. COSC 6397 – Lecture 10 U of H

Novel Applications 3 • Wearable Recognition Systems – Adapt to a specific user and Novel Applications 3 • Wearable Recognition Systems – Adapt to a specific user and be more intimately and actively involved in the user's activities. – Face recognition software can help you remember the name of the person you are looking at. • Useful for Alzheimer's patients. • Smart Systems – Key goal is to give machines perceptual abilities that allow them to function naturally with people. – Critical for a variety of human-machine interfaces. COSC 6397 – Lecture 10 U of H

Why Infrared? 4 • Visible light has no effect on images taken in thermal Why Infrared? 4 • Visible light has no effect on images taken in thermal infrared spectrum. • Even images taken in total darkness are clear in thermal infrared. COSC 6397 – Lecture 10 U of H

Why Infrared? (Contd. . ) 5 • Illumination Invariance – Major problem in visible Why Infrared? (Contd. . ) 5 • Illumination Invariance – Major problem in visible domain. • Uniqueness and Repeatability – Sense thermal patterns of blood vessels under the skin, which transport warm blood throughout the body. – Remain relatively unaffected by aging. – Even identical twins have different thermograms. • Immune from Forgery – Disguises can be easily detected. COSC 6397 – Lecture 10 U of H

Previous Work • Lot of research was done in the visible band but little Previous Work • Lot of research was done in the visible band but little attention was given in the infrared spectrum. • Recent reduction in the cost of infrared cameras and availability of large data sets encouraged active research in infrared face recognition. • Low-Level Models 6 – Directly analyze the image pixels and impose probabilities on the features. – Examples are PCA, ICA, and FDA. – Not good in challenging conditions. • High-Level Models – Synthesize images from 3 D templates of known objects and impose probabilities on transformations. – Template matching approaches. – Computationally expensive. • Our Proposal – Intermediate model which takes advantage of both Low-Level and High-Level models. COSC 6397 – Lecture 10 U of H

Principal Component Analysis • • A D = H x W pixel image of Principal Component Analysis • • A D = H x W pixel image of a face, represented as a vector occupies a single point in D 2 -dimensional image space. Images of faces being similar in overall configuration, will not be randomly distributed in this huge image space. Therefore, they can be described by a low dimensional subspace. Main idea of PCA (cutler 96): 7 – To find vectors that best account for variation of face images in entire image space. – These vectors are called eigen vectors. – Construct a face space and project the images into this face space (eigenfaces). COSC 6397 – Lecture 10 U of H

Eigenfaces Approach - Training • Training set of images represented by 1, 2, 3, Eigenfaces Approach - Training • Training set of images represented by 1, 2, 3, …, M • The average training set is defined by Ψ = (1/M) ∑Mi=1 i • Each face differs from the average by vector Φi = Γi – Ψ • A covariance matrix is constructed as: C = AAT, where A=[Φ 1, …, ΦM] • Finding eigenvectors of N 2 x N 2 matrix is intractable. Hence, find only M meaningful eigenvectors. M is typically the size of the database. COSC 6397 – Lecture 10 8 U of H

Eigenfaces Approach - Training 9 • Consider eigenvectors vi of ATA such that ATAvi Eigenfaces Approach - Training 9 • Consider eigenvectors vi of ATA such that ATAvi = μivi • Pre-multiplying by A, AAT(Avi) = μi(Avi) • The eigenfaces are ui = Avi • A face image can be projected into this face space by Ωk = UT(Γk – Ψ); k=1, …, M COSC 6397 – Lecture 10 U of H

Eigenfaces Approach - Testing 10 • The test image, Γ, is projected into the Eigenfaces Approach - Testing 10 • The test image, Γ, is projected into the face space to obtain a vector, Ω: Ω = UT(Γ – Ψ) • The distance of Ω to each face class is defined by Єk 2 = ||Ω-Ωk||2; k = 1, …, M • A distance threshold, Өc, is half the largest distance between any two face classes: Өc = ½ maxj, k {||Ωj-Ωk||}; j, k = 1, …, M COSC 6397 – Lecture 10 U of H

Eigenfaces Approach - Testing 11 • Find the distance, Є , between the original Eigenfaces Approach - Testing 11 • Find the distance, Є , between the original image, Γ, and its reconstructed image from the eigenface space, Γf, Є2 = || Γ – Γf ||2 , where Γf = U * Ω + Ψ • Recognition process: – IF Є≥Өc then input image is not a face image; – IF Є<Өc AND Єk≥Өc for all k then input image contains an unknown face; – IF Є<Өc AND Єk*=mink{ Єk} < Өc then input image contains the face of individual k* COSC 6397 – Lecture 10 U of H

Limitations of Eigenfaces Approach 12 • Variations in lighting conditions – Different lighting conditions Limitations of Eigenfaces Approach 12 • Variations in lighting conditions – Different lighting conditions for enrolment and query. – Bright light causing image saturation. • Differences in pose – Head orientation – 2 D feature distances appear to distort. • Expression – Change in feature location and shape. COSC 6397 – Lecture 10 U of H

IR Face Recognition – Training Phase 13 COSC 6397 – Lecture 10 U of IR Face Recognition – Training Phase 13 COSC 6397 – Lecture 10 U of H

IR Face Recognition – Test Phase 14 COSC 6397 – Lecture 10 U of IR Face Recognition – Test Phase 14 COSC 6397 – Lecture 10 U of H

Segmentation 15 • Noise in the background may effect the performance of a face Segmentation 15 • Noise in the background may effect the performance of a face recognition system. • Remove the background. • Use thermal information on face to compute the features. • Adaptive Fuzzy Segmentation (kakadiaris 02) – Fuzzy affinity is assigned to spels w. r. t. target object spel. – Affinity is computed as weighted sum of the temperature and the temperature gradient in the neighborhood of the target spel. – Minimal user interaction because of dynamically assigned weights. COSC 6397 – Lecture 10 U of H

Segmentation (Contd. . ) 16 • Fuzzy affinity is calculated by: – Spatial Adjacency: Segmentation (Contd. . ) 16 • Fuzzy affinity is calculated by: – Spatial Adjacency: COSC 6397 – Lecture 10 U of H

Segmentation (Contd. . ) 17 – Temperature homogeneity & gradient: - Temperature of seed Segmentation (Contd. . ) 17 – Temperature homogeneity & gradient: - Temperature of seed c - Mean Temperature - Temperature of seed d - Standard deviation of temperature – Weights: COSC 6397 – Lecture 10 U of H

Problem with Single Seed 18 • Temperatures on face are different at different regions. Problem with Single Seed 18 • Temperatures on face are different at different regions. • If a single seed is chosen in a particular region, then the connectivity stretches only along this region and the segmentation goes wrong. COSC 6397 – Lecture 10 U of H

Multiple Seeds • Solution to this problem is to choose multiple seeds in different Multiple Seeds • Solution to this problem is to choose multiple seeds in different regions on face and merge the resulting segmented parts. • Choose a seed pixel on face wherever there is sharp change in gradient. • Works well even when the subject is wearing glasses. 19 • Robust to variation of poses. COSC 6397 – Lecture 10 U of H

Choosing Multiple Seeds 20 COSC 6397 – Lecture 10 U of H Choosing Multiple Seeds 20 COSC 6397 – Lecture 10 U of H

Assumptions 21 • Merge all resultant segmented regions to form final image. ASSUMPTIONS • Assumptions 21 • Merge all resultant segmented regions to form final image. ASSUMPTIONS • The center of the image contains the pixel from facial region. • The temperatures at all pixels are mapped between 0 and 255. – If this mapped temperature at a pixel is between 175 - 200, it is classified to be in blue region. – If this mapped temperature at a pixel is between 200 - 225, it is classified to be in pink region. – If this mapped temperature at a pixel is between 225 - 255, it is classified to be in cyan region. COSC 6397 – Lecture 10 U of H

Feature Extraction 22 • The segmented facial image is divided into its spectral components Feature Extraction 22 • The segmented facial image is divided into its spectral components using Gabor filters. • The resultant Gabor filtered images are modeled using Bessel models. • The Gabor filter bank is given by: COSC 6397 – Lecture 10 U of H

Gabor Filter Bank • Example Gabor filter bank with 3 scale values and 4 Gabor Filter Bank • Example Gabor filter bank with 3 scale values and 4 orientation values: COSC 6397 – Lecture 10 23 U of H

Spectral Components 24 COSC 6397 – Lecture 10 U of H Spectral Components 24 COSC 6397 – Lecture 10 U of H

Bessel Parameters • 25 Each segmented image in training set is convolved with the Bessel Parameters • 25 Each segmented image in training set is convolved with the filters in Gabor filter bank to obtain Gabor filtered images. • The filtered images are modeled using Bessel parameters: SK – Sample Kurtosis SV – Sample Variance COSC 6397 – Lecture 10 U of H

Sample Variance and Kurtosis 26 • Sample Variance is the measure of the “spread” Sample Variance and Kurtosis 26 • Sample Variance is the measure of the “spread” of the distribution. • Sample Kurtosis is the measure of the “peakedness” or “flatness”. Sample Kurtosis, COSC 6397 – Lecture 10 U of H

Bessel Model 27 • Using the bessel parameters p and c, the filtered image Bessel Model 27 • Using the bessel parameters p and c, the filtered image I(j)(x, y) is modeled as: (p) is gamma function COSC 6397 – Lecture 10 Iv(z) is modified bessel function of first kind given by: U of H

Bessel Model 28 COSC 6397 – Lecture 10 U of H Bessel Model 28 COSC 6397 – Lecture 10 U of H

Performance of Bessel K Forms 29 • Kullback-Leiber divergence: – observed marginal density – Performance of Bessel K Forms 29 • Kullback-Leiber divergence: – observed marginal density – Estimated Bessel Form KL div=0. 0013 KL div=0. 0027 COSC 6397 – Lecture 10 KL div=0. 0055 KL div=0. 0058 U of H

Comparing IR Images 30 • • Images modeled into Bessel parameters can be compared Comparing IR Images 30 • • Images modeled into Bessel parameters can be compared by: L 2 -metric between two Bessel forms f(x; p 1, c 1) and f(x; p 2, c 2) in D: COSC 6397 – Lecture 10 U of H

Hypothesis Pruning 31 • Applying a high-level classifier on entire database is computationally very Hypothesis Pruning 31 • Applying a high-level classifier on entire database is computationally very expensive. • Pruning of hypotheses can be achieved by using Bessel parameters (anuj 01). • Helps in short listing best matches. • Bessel parameters for images in database can be computed offline which helps in saving a lot of computation time. COSC 6397 – Lecture 10 U of H

Hypothesis Pruning (Contd. . ) 32 • Define a probability mass function on the Hypothesis Pruning (Contd. . ) 32 • Define a probability mass function on the database A: (D=0. 3 for Equinox dataset) (p(j)obs, c(j)obs) – observed Bessel parameters for test image I(j) (p(j) , s, c(j) , s) – estimated Bessel parameters which can be computed offline • Images in database A with P 1( |I) greater than a specific threshold value are short listed as best matches. COSC 6397 – Lecture 10 U of H

Hypothesis Pruning (Contd. . ) • Shortlist the subjects of A with P 1( Hypothesis Pruning (Contd. . ) • Shortlist the subjects of A with P 1( /I) greater than a specific threshold: COSC 6397 – Lecture 10 33 U of H

Pruning Algorithm 34 COSC 6397 – Lecture 10 U of H Pruning Algorithm 34 COSC 6397 – Lecture 10 U of H

Classification 35 • Bayesian target recognition (anuj 00) searches for the target hypothesis with Classification 35 • Bayesian target recognition (anuj 00) searches for the target hypothesis with largest posterior probability given by: – Likelihood: : Variance of test image d : dimension of image (2 in this case) – Apriori is same for all images in database (for database of n images, it is 1/n for each image). COSC 6397 – Lecture 10 U of H

Experiments • Equinox Database: 36 www. equinoxsensors. com • Image frame sequences were acquired Experiments • Equinox Database: 36 www. equinoxsensors. com • Image frame sequences were acquired at 10 frames/sec while the subject was reciting the vowels ‘a’, ’e’, ’i’, ’o’, ’u’. COSC 6397 – Lecture 10 U of H

Results – ROC Curves 37 Correct Positive : Test image is in the database Results – ROC Curves 37 Correct Positive : Test image is in the database and is correctly recognized. False Positive : Test image is not in the database, but is recognized to be an image of the database Negatives : Test images that are not in the database. COSC 6397 – Lecture 10 U of H

Results – Precision & Recall 38 COSC 6397 – Lecture 10 U of H Results – Precision & Recall 38 COSC 6397 – Lecture 10 U of H

Conclusion 39 • We came up with a face recognition approach which is computationally Conclusion 39 • We came up with a face recognition approach which is computationally inexpensive and at the same time good in challenging conditions. • The features of all images in database can be computed offline and stored for future use. This saves lot of computation time. • We improved the performance of classifier by removing background noise of pruned hypothesis using adaptive fuzzy connectedness based image segmentation. COSC 6397 – Lecture 10 U of H

References 40 • • [anuj 01] A. Srivastava, X. W. Liu, B. Thomasson, and References 40 • • [anuj 01] A. Srivastava, X. W. Liu, B. Thomasson, and C. Hesher, "Spectral Probability Models for IR Images with Applications to IR Face Recognition, " in Proceedings 2001 IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Kauai, HI, Dec 14. [cutler 96] R. Cutler, “Face recognition using infrared images and eigenfaces”, website, http: //www. cs. umd. edu/rgc/face. htm, 1996. [anuj 00] A. Srivastava, M. I. Miller, and U. Grenander, “Bayesian automated target recognition, " Handbook of Image and Video Processing, Academic Press, pp. 869 -881, 2000. [kakadiaris 02] A. Pednekar, I. A. Kakadiaris, U. Kurkure. Adaptive fuzzy connectedness-based medical image segmentation. In Proc. of the Indian Conf. on Computer Vision, Graphics, and Image Processing (ICVGIP 2002), pp. 457 -462, Ahmedabad, India, December 16 -18 2002. COSC 6397 – Lecture 10 U of H