Biometrics and Sensors Venu Govindaraju CUBS University at

Biometrics and Sensors Venu Govindaraju CUBS, University at Buffalo govind@buffalo. edu

Organization § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Research at UB § Multimodal Identification § Biometrics § Fingerprint § Signature § Hand Geometry § Sensors § Materials and Light Sources § Analog VLSI and Optical Detectors § Packaging and Reliability Engineering

Scope of Research In Biometrics State of the art Research Problems Fingerprint 0. 15% FRR at 1% FAR (FVC 2002) §Fingerprint Enhancement §Partial fingerprint matching Face Recognition 10% FRR at 1% FAR (FRVT 2002) §Improving accuracy §Face alignment variation §Handling lighting variations Hand Geometry 4% FRR at 0% FAR (Transport Security Adminstration Tests) §Developing reliable models §Identification problem Signature Verification § 1. 5% (IBM Israel) §Developing offline verification systems §Handling skillful forgeries Chemical Biometrics No open testing done yet §Development of sensors §Materials research

Biometrics § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Conventional Security Measures § Token Based § Smart cards § Swipe cards § Knowledge Based § Username/password § PIN § Disadvantages of Conventional Measures § Tokens can be lost or misused § Passwords can be forgotten § Multiple tokens and passwords difficult to manage

Biometrics § Definition § Biometrics is the science of verifying and establishing the identity of an individual through physiological features or behavioral traits § Examples § Physical Biometrics § Fingerprint, Hand Geometry, Iris, Face § Behavioral Biometrics § Handwriting, Signature, Speech, Gait § Chemical Biometrics § DNA, blood-glucose

Fingerprint Verification § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Fingerprint Verification Fingerprints can be classified based on the ridge flow pattern Fingerprints can be distinguished based on the ridge characteristics

Fingerprint Image Enhancement • Preprocessing • Enhancement • Feature Extraction • Matching High contrast print Typical dry print Low contrast print Typical Wet Print

Traditional Approach • Preprocessing • Enhancement • Feature Extraction • Matching Local Orientation (x, y) Gradient Method Enhancement Frequency/Spatial Local Ridge Spacing F(x, y) Projection Based Method

Fourier Analysis Approach • Preprocessing • Enhancement • Feature Extraction Energy Map E(x, y) • Matching FFT Analysis Orientation Map O(x, y) Ridge Spacing Map F(x, y) FFT Enhancement

Fourier Analysis – Applied to fingerprints Fingerprint ridges can be modeled as an oriented wave Local ridge orientation Local ridge frequency

Fourier Analysis –Energy Map • Preprocessing • Enhancement • Feature Extraction • Matching Original Image Energy Map Thresholded Map

Fourier Analysis – Frequency Map • Preprocessing • Enhancement • Feature Extraction • Matching Original Image Local Ridge Frequency Map

Fourier Analysis-Orientation Map • Preprocessing • Enhancement • Feature Extraction • Matching Original Image Local Ridge Orientation Map

FFT Based Enhancement • Preprocessing • Enhancement • Feature Extraction • Matching Original Image Enhanced Image

Common Feature Extraction Methods • Preprocessing • Thinning-based Method • Enhancement • Thinning produces artifacts • Feature Extraction • Shifting of Minutiae coordinates • Matching • Direct Gray-Scale Extraction Method • Difficult to determine location and orientation • Binarized Image is noisy.

Chaincoded Ridge Following Method • Preprocessing • Enhancement • Feature Extraction • Matching

Minutiae Detection • Preprocessing • Enhancement § • Feature Extraction • Matching § § Several points in each turn are detected as potential minutiae candidate One of each group is selected as detected minutiae. Minutiae Orientation is detected by considering the angle subtended by two extreme points on the ridge at the middle point.

Pruning Detected Minutiae • Preprocessing • Enhancement • Feature Extraction • Matching § Ending minutiae in the boundary of fingerprint images need to be removed with help of FFT Energy Map § Closest minutiae with similar orientation need to be removed

Secondary Features • Preprocessing • Enhancement • Feature Extraction • Matching § § Pure localized feature Derived from minutiae representation Orientation invariant Denote as (r 0, r 1, δ 0, δ 1, ) § r 0, r 1: lengths of MN 0 and MN 1 § δ 0, δ 1: relative minutiae orientation w. r. t. M § : angle of N 0 MN 1

Dynamic Tolerance Areas • Preprocessing § • Enhancement § § • Feature Extraction • Matching Tolerance Area is dynamically decided w. r. t. the length of the leg. Longer leg: Tolerates more distortion in length than the angle. Shorter leg: tolerates less distortion in length than the angle. B A O Dynamic tolerance Dynamic Windows

Feature Matching • Preprocessing • Enhancement • Feature Extraction • Matching 1. For each triangle, generate a list of candidate matching triangles 2. To recover the rotation between the prints. Find the most probable orientation difference 3. Apply the results of the pruning and match the rest of the points based on the reference points established.

Validation • Preprocessing • Enhancement • Feature Extraction • Matching 1. For each triangle, generate a list of candidate matching triangles 2. To recover the rotation between the prints. Find the most probable orientation difference 3. Apply the results of the pruning and match the rest of the points based on the reference points established. OD=0. 7865°

Minutia Matching • Preprocessing • Enhancement • Feature Extraction • Matching 1. For each triangle, generate a list of candidate matching triangles 2. To recover the rotation between the prints. Find the most probable orientation difference 3. Apply the results of the pruning and match the rest of the points based on the reference points established

Data Sets Fig(a) Sensors and technology used in acquisition Fig(b) Paired fingerprints Fig(c) Database sets

Preliminary Results • Min Total Error = 1. 16% Threshold • ERR • FRR at 0 FAR = 1. 0% = 5. 0% FRR State of the art • Min Total Error = 0. 19% • FRR at 0 FAR = 0. 38%

Signature Verification § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Signature Verification Online Signature verification Off line Signature Verification

Preprocessing § Preprocessing • Preprocessing Make signature invariant to scale, translation and rotation. • Template generation • Matching (-170)- (-125) (-3. 0)- (4. 0) mean-std norm. Smoothing Resampling -1. 5 -3. 5 0 -160

Template Generation- Challenges • Preprocessing § • Template generation Extracting features. Usually we can not expect more than 6 genuine signatures for training for each subject. This is unlike handwriting recognition • Matching § Decide the consistent features. There are over 100 features for signature, such as Width, Height, Duration, Orientation, X positions, Y positions, Speed, Curvature, Pressure, so on.

Matching – Similarity Measure • Preprocessing § Simple Regression Model • Template generation • Matching Y = (y 1 , y 2 , …, yn) X = (x 1 , x 2 , …, xn) Similarity by R 2 : 91% R 2 = Similarity by R 2 : 31%

Traditional Regression approach • Preprocessing § • Template generation • Matching § Advantages: Invariant to scale and translation. Similarity (Goodness-of-fit) makes sense. Disadvantages: One-one alignment, brittle. One-One alignment Dynamic alignment

Dynamic Regression approach(1) • Preprocessing • Template generation • Matching ( y 2 is matched x 2, x 3, so we extend it to be two points in Y sequence. ) Similarity = R 2 Where (x 1 i, y 1 i, v 1 i) are points in the sequence And a, b, c are the weights, e. g. , 0. 5, 0. 25 • DTW warping path in a n-by-m matrix is the path which has min cumulative cost. • The unmarked area is the constrain that path is allowed to go.

Offline Signature Verification • Shapes can be described using structural or statistical features • We use an analytical approach that uses the attributes of structures. Extracting structural features

Attributes of structural features Statistical analysis of the feature attributes Attributes of structural features

Hidden Markov Models and SFSA • Obtaining a stochastic model • Outgoing transitional probabilities • The occurrence of the structural features can be modeled as a HMM • The HMM can be converted to a SFSA by assigning observation and probability to the transitions instead of to the states

Hand Geometry § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Hand Geometry § § Used where Robustness, Low cost are the concerns. Comparatively less accurate. Combination with other Biometric techniques, increases accuracy. Sufficient for verification where finger print use may infringe on privacy.

Feature Extraction § § A snapshot of the top and side views of the user’s right hand gives the contours outlining the hand. Features necessary to identify the hand are extracted from these contours. Using simple image processing techniques, the contours of the set of two images of the hand are obtained. Hand-verification is done by correlating these features. Research: New features and algorithms for better discrimination between two hands.

Multimodal Biometrics § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Combination of biometric matchers Combination of the matching results of different biometric features provides higher accuracy. Fingerprint matching Signature matching Hand geometry matching Alice 26 Bob 12 : : Alice 0. 31 Bob 0. 45 : : Alice 5. 54 Bob 7. 81 : : Combination algorithm Alice 0. 95 Bob 0. 11 : :

Sequential combination of matchers Fingerprint matching Combination algorithm 1 No Signature matching Hand geometry matching Desired confidence achieved? Yes Combination algorithm 2 No Desired confidence achieved? Combination algorithm 3 Yes Alice 0. 95 Bob 0. 11 : :

Securing Biometric Data § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Signature Verification Hand Geometry Multimodal biometrics Securing Biometric Data Sensors and Devices

Securing password information It is impossible to learn the original password given stored hash value of it.

Securing fingerprint information Wish to use similar functions for fingerprint data:

Obstacles in finding hash functions Fingerprint space f 1 f 2 Hash space h h(f 1) h(f 2) Since match algorithm will work with the values of hash functions, • similar fingerprints should have similar hash values • rotation and translation of original image should not have big impact on hash values • partial fingerprints should be matched

Sensors and Devices § § § § Biometrics and Sensor research at UB Biometrics Fingerprint Verification Securing Biometric Data Signature Verification Hand Geometry Sensors and Devices

Sensors and Biometrics Fingerprint §Optical Sensors §Capacitive Sensors §Thermal Sensors §Ultrasound Sensors Signature §Digitizer Tablet §Digitizer Pen §Offline scanning Face Recognition §Optical Digital camera §Thermal cameras Chemical Biometrics §Sensor Arrays §Smart Devices (Research at UB)

Sensors Detector System • • CMOS CCD’s Photodiodes Image Processing • Biosurfaces - Biofouling • Bioinspired Pattern Recognition • Biomimetics – Artificial Vision, Smell. • Bioinspired Super Correlator Analyte • Tissues • Cells Sensing • Proteins Layer • DNA and RNA • Organic and Inorganic Dyes • Molecular Imprinting • Light Sources (OLEDs, Lasers) Stimulator • Signal Generators and Support System • Driver Circuits • Power Supply c) Device b) Enabling Technologies • Biosurfaces – Biofouling • Immobilization and Stabilization • Transduction mechanism • Multi-Analyte detection • Photonic Bandgap (PBG) Resonators • Evanescent Wave Devices (PBG) • Biosurfaces – Biofouling • Nano-LEDs • Bioinspired Photovoltaics, Biofuel Cells • Environmental Testing • Low Power Light Sources a) Fundamental Knowledge

Sensor Components Stimulator and Support System Sensing Layer Detector System Blocking Filter Output Device

CMOS Integrated Sensor System

Sensor System Components 60 mm 1. 2 mm thick

Protein Imprinted Xerogels with Integrated Emission Sites Protein Analyte * * * * ** Response (%) PIXIES • The sensors selectively respond to Ovalbumin • Orders of magnitude greater than other components • Each site can individually respond to different analytes

Summary § A unique collaborative initiative that enables state-of-the-art Biometric Science and Technology § Creating a multi-disciplinary environment attracting faculty and students from engineering and sciences § Preparing and educating future Biometric Scientists and Engineers § Targeting all the aspects of Biometrics from authentication to materials and including them into a packaged device

Websites § www. cubs. buffalo. edu § www. photonics. buffalo. edu § www. cedar. buffalo. edu § www. packaging. buffalo. edu Acknowledgements Financial support of: § § National Science Foundation (NSF) Office of Naval Research (ONR) Calspan UB Research Center (CUBRC) University at Buffalo Center for Advanced Technology (UBCAT)

Thank You