Automated Rivet Inspection System for Aging Aircrafts Unsang

Automated Rivet Inspection System for Aging Aircrafts Unsang Park, Lalita Udpa, George C. Stockman Computer Science and Engineering Michigan State University 1

Contents Nondestructive Inspection (NDI) Magneto-optic Imager in NDI Motion-based Filtering (MBF) Real-time implementation of MBF Automated rivet inspection system Rivet detection Rivet classification Results and conclusions Future work 2

Nondestructive Inspection for Aircrafts Detect subsurface defects Seam Rivet Crack 3

Nondestructive Inspection for Aircrafts Increase service life of airplane Prevent disasters Aloha Airlines B-737 -200 lost part of its front fuselage during a flight in Hawaii, 1985 4

Magneto-optic Imager (MOI) Eddy current excitation Magneto-optic sensing Imaging CCD Camera MO Sensor Bias Coil Light Source Polarizer Analyzer Induction Sheet Sample 5

Magneto-optic Imager (cont. ) Produce real-time analog images of inspected part Images both surface breaking and subsurface cracks Easy to interpret with minimal training Applicable both on conducting samples as well as composites by tagging with ferromagnetic particles 6

Sample MOI Images Crack along seam Corrosion dome Crack between two rivets, Radial crack on a rivet 7

Drawback of MOI images MOI image contains serpentine pattern noises due to the magnetic domain walls in magneto-optic sensor Signals due to domain walls Rivet 8

Motion-based Filtering (MBF) Additive Frame Subtraction In-5 In-4 D 5 In-3 D 4 In-2 D 3 In-1 D 2 In D 1 Sn Moving direction of objects Moving direction of MOI 9

Motion-based Filtering (cont. ) Input Image Preprocessing RGB to Gray Additive Frame Subtraction Threshold Post processing Median Filter Stretch Output Image 10

MB filtered images Original Filtered 11

MB filtered images (cont. ) Original Filtered 12

Real-time implementation of MBF Experimental setup for proof of concept Record to VHS Frame grabber Play on a Video player Collects MOI image Data Record to movie file noise Web camera Play on a PC monitor 13

Real-time implementation of MBF (cont. ) Data transfer rate 4. 6 Mbytes /sec ( 320 240 pixels 16 bit 30 fps ) Data are down sampled as the input images are dropped while an image is processed Diagram of real-time Motion-based Filtering MOI Real-time MBF Sensing Max RGB to Gray Imaging Displaying Subtract Max Stretch Median Filtering Threshold 14

Real-time implementation of MBF (cont. ) Optimizing MBF algorithm in C++ Image capture: 20 ms RGB to Gray: 20 ~ 23 ms Additive Frame Subtraction: 1~2 ms/image Threshold: 1 ~ 2 ms Median Filter: 200 ~ 250 ms Stretch: 1 ~ 2 ms Output Image 15

Real-time implementation of MBF (cont. ) RGB to Gray conversion Normal 320*240 16 bit image Table Lookup Table build time 20~23 ms 1~2 ms 5~6 ms (1 M bytes) Additive frame subtraction MAX(I 1 -I 3, I 2 -I 3) MAX(I 1, I 2) - I 3 Median filter 5 by 5 7 by 7 MATLAB 50 ms 88 ms Modified Quick Sort (C++) 200 ~ 250 ms 400 ~ 450 ms Moving Median with Sorting (C++) 100 ~ 150 ms 150 ~ 200 ms Moving Median with Histogram (C++) 20 ~ 25 ms 20 ~ 40 ms 16

Real-time implementation of MBF (cont. ) Execution time of MBF algorithm in C++ Before optimization Intel 2 GHz, C++ After optimization Intel 2 GHz, C++ Capture 20 ms RGB to gray 20 ~ 23 ms 1 ~ 2 ms Subtraction (x 10) Max (x 10) Threshold 1 ~ 2 ms (x 10) 1~2 ms 1 ~ 2 ms (x 10) 1 ~ 2 ms Median Filter (3 x 3) 200 ~ 250 ms 20 ~ 30 ms Stretch 1 ~ 2 ms Total 262 ~ 337 ms 53 ~ 76 ms 17

Drawbacks of current MOI inspection No measure for quantitative interpretation Data interpretation is subjective Manual inspection by human operator (more than 10 hours per airplane) Expensive labor cost Error due to fatigue 18

Automated MOI inspection system PRI Research and Development Corporation (PRI) Developing and improving magneto-optic imager (MOI) Michigan State University, ECE department Image processing algorithm for filtering and classification Boeing Phantom Works Self-guided, suction cup robot – crawls over airplane skin 19

Automated MOI inspection (cont. ) Currently focusing on radial cracks on rivets Quantification of defects in MOI images Implementing real-time rivet inspection algorithm Motion-based Filtering Rivet detection Rivet classification 20

Rivet detection Hough transformation-based method Circular Hough transformation Morphological operation-based method Segment out each rivet Obtain center and radius, c 1 , r 1 Erode rivet with a circle of radius r 1 yes, r 1 -1 Area(rivet) = 0 no Obtain center and radius, c 2, r 2 21

Rivet detection (cont. ) Hough transformation Original Morphological operation 22

Rivet classification Two-pass Hough transformation 1 st pass – Rivet detection 2 nd pass – Blob detection good Original image Filtered image - After 1 st pass bad After 2 nd Pass 23

Rivet classification (cont. ) Bayesian classifier Feature selection Original image Hough transformation Morphological op. 24

Off-line test Training 10 normal, 10 defective rivet images Obtain mean and variance of feature f 1 Testing 222 rivet images including 66 defective rivet images Two-pass Hough - Bayes Morph. - Bayes 25

Experimental Results Accuracies of three algorithms Rivet detection Inspection algorithm False negative False positive Two-pass Hough 1/242 0/242 90% (200/222) Hough-Bayes 1/242 0/242 96% (214/222) Morph. -Bayes 1/242 99% (220/222) Rivet classification 26

Conclusions MB filtered image is optimal in image processing for automated rivet inspection Morphological operation-based rivet detection is superior to Hough-based rivet detection both for execution time and accuracy Bayesian classifier is superior to Hough-based classifier Radial crack detection on rivets showed 99% accuracy in off-line test 27

Future work Implement MBF and rivet inspection algorithms on the Digital Signal Processing (DSP) board Improve robustness of the algorithms with the feedback from field test Develop MOI inspection algorithms for other types of defects in aircrafts 28