Ear biometrics Gökhan Şengül Contents n n

Ear biometrics Gökhan Şengül

Contents n n Using ear shape as biometrics Three types of ear biometrics n n n Pictures Earmarks Thermograms Two cases Application scenarios and possible errors

Biometrics are unique physical or behavioral characteristics of an individual which can be measured and thus compared to accurately verify or identify an individual. http: //www. biopay. com/

Ideal biometric n n universal: each person should possess the characteristics unique: no two persons should share the characteristics permanent: the characteristics should not change collectable: easily presentable to a sensor and quantifiable

Ear shape n n n physical biometric, which is characterized by the shape of the outer ear, lobes, curves, surfaces, geometric measurements, and bone structure unique enough? new biometric, not widely used yet

Ear Most important parts in ear shape identification are • Foseta • Antitrago • Helix

Ear

Ear recognition - advantages n n n The structure of the ear has been observed to be stable despite aging, and ear growth is almost linear after the age of four The ear, unlike other facial features, is minimally impacted by changes in facial expression Image acquisition does not involve explicit contact with the sensor

Ear recognition- disadvantages n n n Ears are smaller than e. g. faces reduced spatial resolution Ears are not as variable as e. g. faces We have almost none adjectives to describe ears: we can recognize people from faces but can we recognize them from ears?

Same ear can look different…

Permanence of biometrics Bromba Gmb. H

Ear recognition system n Ear detection module n n Feature extraction module n n Extracts discriminative features from the ear Matching module n n localizes the position and spatial extent of the ear in an image; Extracts the features extracted from two ear images and generates a match score Decision module n processes the match score(s) and establishes the identity of the subject

Ear Detection n Templete Matching n n a template of a typical ear is constructed and is matched with each location in the query image. The location giving the highest score is considered as the region containing the ear. The template may consist of an edge image of the ear or a set of descriptors extracted from the ear such as the response to a set of filters or a histogram of shape curvatures in case a 3 D image of the ear is being used for recognition.

Ear Detection n Model-based detection n n A model-based detection technique assumes certain characteristics of the shape of the ear and tries to find regions that manifest such characteristics. The shape of the helix, for example, is usually elliptical so a generalized Hough transform tuned for detecting ellipses can be used to locate the ear in an edge image.

Ear Detection n Morphological-operator-based detection n n Since the structure of the ear is usually more intricate than the structure of the remaining region in a profile face image, morphological transformations such as the Top-hat transformation can be used. A Top-hat transformation essentially subtracts a morphologically smoothened version of an image from itself, thereby highlighting finer details.

Ear Detection n Face-geometry-based detection n Since in a profile image the nose can be easily detected as the point with high curvature, it is possible to constrain the search for the ear in an appropriate location relative to the nose.

Ear Recognition n Subspace analysis-based techniques n projecting the ear image onto a set of principal directions is an effectiveway to obtain a salient and compact representation of an ear. Subspace projection techniques such as PCA, ICA, and LDA have been successfully used in literature for matching ear images.

Ear Recognition n Sparse-representation-based techniques n Optimization techniques that minimize L-1 norm of the distance vector between the transformed query and all the transformed templates in a database have been shown to provide high recognition accuracy in object recognition studies. This technique has also been successfully used for ear recognition.

Ear Recognition n Point-set-matching-based techniques n Scale Invariant Feature Transform (SIFT) is a well known technique for matching two images where a set of salient points can be reliably and repeatably extracted from them. In order to match two images using SIFT features, corner points are detected from two images and matched based on image gradient-based features extracted from the neighborhood region of each point.

Ear Recognition n Point-set-matching-based techniques

Ear Recognition n Geometric-measurements-based techniques n Features obtained by measuring certain geometric characteristics of the ear can also be used as a set of discriminative features. As an example, the centroid of an ear image obtained from its edge image can be used as a center to draw concentric circles with pre-specified radii. Various measurements, such as number of points on a circle intersecting the edge image or distance between two consecutive intersections, can be used as a feature vector.

Ear Recognition n Transformation-based techniques n n Various image transformation techniques such as Fourier transform or wavelet transform can also be applied to extract discriminative features from an ear image. Fourier transform can also be applied in order to obtain a rotation and translation invariant representation of the ear

Ear Recognition n 3 D techniques n n In some scenarios, acquiring a 3 D rendition of the ear entity may be possible. 3 D images offer depth information that can be used in conjunction with the 2 D texture information to improve the recognition accuracy. In the case of 3 D ear images, local histograms of shape curvature values can be used to match two ear images.

3 types of ear biometrics n n n biometrics from photo of ear: most researched, “ear marks”: interesting in crime detection thermogram pictures: a new possibility

Methods using pictures (1/4) n Moreno et al. (1999) n n multiple identification method, which combines the results from several neural classifiers using feature outer ear points, information obtained from ear shape and wrinkles, and macro features extracted by compression network introduce three different classification techniques for outer ear or auricle identifying.

Methods using pictures (2/4) n Burge and Burger (1998, 2000) n n automating ear biometrics with Voronoi diagram of its curve segments. a novel graph matching based algorithm for authentication, which takes into account the possible error curves, which can be caused by e. g. lightning, shadowing and occlusion.

Methods using pictures (3/4) n Hurley, Nixon and Carter (2000 a, 2000 b) n n n force field transformations for ear recognition. the image is treated as an array of Gaussian attractors that act as the source of the force field according to the researchers this feature extraction technique is robust and reliable and it possesses good noise tolerance.

Methods using pictures (4/4) n Victor, Chang, Bowyer, Sarkar (at least 2 publications in 2002 and 2003) n n principal component analysis approach comparison between ears and faces This method is presented later with 2 cases.

Earmarks n n when the ear is pressed against some hard material, e. g. glass, the result is called earmark used in forensic research at least 4 criminals are judged because of earmarks in decade of 1980 in England in The Netherlands later it was decided by a Dutch court that earmarks are not unique and reliable enough

Ear thermogram used for removing “non-ear”, e. g. hair and other obstacles in this case hair is between 27. 2 and 29. 7ºC while the outer ear areas range from 30. 0 to 37. 2 ºC Thermogram of an ear. Image provided by Brent Griffith, Infrared Thermography Laboratory, Lawrence Berkeley, National Laboratory. (Burge et al. , 1998)

Case 1: an evaluation of face and ear biometrics n n n Done by Victor, B. , Bowyer, K. , and Sarkar S, published in Proceedings of International Conference on Pattern Recognition, August 2002 The used method is principal component analysis (PCA) and the design principle is adopted from the FERET methodology Null hypothesis: there is no significant performance difference between using the ear or face as a biometric

PCA Method

Points for normalization

Tests of research by Victor et al. n For faces: 1. 2. 3. n Same day, different expression Different day, similar expression Different day, different expression For ears: 1. 2. 3. Same day, opposite ear Different day, same ear Different day, opposite ear

Same day, different expression or opposite ear

Different day, similar expression or same ear

Different day, different expression or opposite ear

Victor et al. research result Experiment # Face/Ear compared 1 Same day, different opposite ear expression 2 Different day, similar same ear expression 3 Differet day, Different day, different opposite ear expression Expected Result Greater variation in Face performs expressions than ears; ears better perform better Greater variation in Face performs expression across days; ears better perform better Greater variation in face Face performs expression than ear; ears better perform better

Case 2: Ear and Face images n n n ”Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics” by Chang, K. , Bowyer, K. W. , Sarkar, S. , and Victor, B. Published in IEEE Transactions on Pattern Analysis and Machine Intelligence on September 2003. Hypothesis: (i) ear provide better biometric performance than images of the face and (ii) exploring whether a combination of ear and face images may provide better performance than either one individually

Images used in research by Chang et al. (2003) Same kinds of sets for faces, too. PCA, FERET

Tests for the research day variation 1. ¨ other conditions constant different lightning condition 2. ¨ taken in the same day in the same session pose variation 3. ¨ 22. 5 degree rotation, other conditions constant, taken in the same day Combined face, ear and face+ear What do you think the result was?

Day variation test

Different lightning conditions

Pose variation (22. 5 degree rotation)

Results n n In this research face biometrics seem to be better in constant conditions, ear biometrics in changing conditions Multimodal biometrics face plus ear gives the best results, why not use them?

Ear shape applications n n n currently there are no applications, which use ear identification or authentication crime investigation is interested in using ear identification active ear authentication could be possible in different scenarios

Application scenarios n Application scenarios are quite traditional for biometrics: n n collecting child from daycare ATM any other active identification passive n e. g. when trying to solve a crime and there’s a picture of ear in the tape of a surveillance camera n e. g. gas station robbery in The Netherlands

Biometric suitability for authentication purpose Bromba Gmb. H

Error possibilities in ear recognition

False rates in ear identification n n FRR = false reject rate FAR = false acceptance rate Normally one of these is tried to minimize.

Improving the FRR with ear curve widths, an example width of an ear curve corresponding to the upper Helix rim better results

Removal of noise curves in the inner ear Graph model (Burge et al. ) and false curves because of e. g. oil and wax of the ear.

Possibilities to enhance ear biometrics n n n Using accurate measurements, e. g. ear curve and upper helix rim Removing noise curves Thermograms removal of obstacles Better quality cameras more accurate pictures Combined biometrics

Challenges in ear recognition n Large-scale public evaluation of ear recognition has not been conducted. There are no commercial ear recognition system The performances of ear recognition algorithms have been tested on some standard ear datasets. Experiments suggest that ear images obtained under controlled conditions can result in good recognition accuracy

References (1/2) n n n n Bamber, D. Prisoners to appeal as unique ‘earprint’ evidence is discredited. Telegraph Newspaper (UK). Updated 02/12/2001 [Retrieved October 3, 2003] From: http: //portal. telegraph. co. uk/news/main. jhtml? xml=/news/2001/12/02/nearp 02. xml Bromba Gmb. H, Bioidentification Frequently Asked Questions. Updated 2003 -09 -12 [Retrieved October 28, 2003] From: http: //www. bromba. com/faq/biofaqe. htm. Burge, M. and Burger, W. Ear Biometrics. In A. Jain R. Bolle and S. Pankanti, editors, BIOMETRICS: Personal Identification in a Networked Society, pp. 273 -286. Kluwer Academic, 1998. Burge, M. and Burger, W. Ear Biometrics in Computer Vision. In the 15 th International Conference of Pattern Recognition, ICPR 2000, pp. 826 -830. Carreira-Perpinan, M. A. Abstract from MSc thesis Compression neural networks for feature extraction: Application to human recognition from ear images, Technical University of Madrid. 1995. [Retrieved October 16, 2003]. From http: //www. dcs. shef. ac. uk/~miguel/papers/msc-thesis. html. Chang, K. , Bowyer. K. W. , Sarkar, S. , Victor, B. Comparison and Combination of Ear and Face Images in Appearance-Based Biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, September 2003, pp. 1160 -1165. Forensic Evidence News: Ear Identification. Version updated Nov. 1, 2000. [Retrieved October 3, 2003] From http: //www. forensic-evidence. com/site/ID/IDear. News. html.

References (2/2) n n n n Hoogstrate, A. J. , Van den Heuvel, H. , Huyben, E. Ear Identification Based on Surveillance Camera’s Images. Version updated May 31, 2000. [Retrieved October 7, 2003] From: http: //www. forensicevidence. com/site/ID/IDear. Camera. html. Hurley, D. J. , Nixon, M. S. , Carter, J. N. Automated Ear Recognition by Force Field Transformations in Proceedings IEE Colloquium: Visual Biometris (00/018), 2000 a, pp. 8/1 -8/5. Hurley, D. J. , Nixon, M. S. , Carter, J. N. A New Force Field Transform for Ear and Face Recognition. In Proceedings of the IEEE 2000 International Conference on Image Processin ICIP 2000 b, pp. 25 -28. Jain, A. , Hong, L. , Pankati, S. Biometric Identification. Communications of the ACM, February 2000/Vol. 43, No. 2, pp. 91 -98. Moreno, B. , Sánchez, Á. , Vélez. J. F. On the Use of Outer Ear Images for Personal Identification in Security Applications. IEEE 33 rd Annual International Carnahan Conference on Security Technology, 1999, pp. 469 -476. Morgan, J. Court Holds Earprint Identification Not Generally Accepted In Scientific Community, State v. David Wayne Kunze. 1999. [Retrieved September 9, 2003]. From: http: //www. forensicevidence. com/site/ID/ID_Kunze. html Ratha, N. K. , Senior, A. , Bolle, R. M. Automated Biometrics in Proceedings of International Conference on Advances in Pattern Recognition, Rio de Janeiro, Brazil, March 2001. Victor, B. , Bowyer, K. , Sarkar, S. An evaluation of face and ear biometrics in Proceedings of International Conference on Pattern Recognition, pp. 429 -432, August 2002.

Questions? Thank you for your attention! G. ŞENGÜL