Memorial University of Newfoundland Faculty of Engineering

Memorial University of Newfoundland Faculty of Engineering & Applied Science Engineering 7854 Industrial Machine Vision INTRODUCTION TO MACHINE VISION II Prof. Nick Krouglicof Memorial University of Newfoundland 3/15/2018

Presentation Outline 1. Machine vision systems for mechanical metrology • • Algorithms for camera calibration Development of 3 D vision systems for MI Flume Tank 2. Industrial Applications of Machine Vision • • High speed, line scan camera-based inspection system for the food processing industry Vision based inspection of liquid crystal display (LCD) modules Memorial University of Newfoundland 3/15/2018

A systematic approach to the calibration of machine vision systems for industrial metrology • In the field of machine vision, camera calibration refers to the experimental determination of a set of parameters which describe the image formation process for a given analytical model of the machine vision system. • Ideally, camera calibration is performed without specialized optical equipment, without modifications to the hardware, and without a priori knowledge of the vision system. • Most calibration techniques are based on the observation of planar (2 D) targets with a large number of control points. Memorial University of Newfoundland 3/15/2018

A systematic approach to the calibration of machine vision systems for industrial metrology The machine vision parameters which must be identified include : a) b) c) d) e) The scale factor The frame buffer coordinates of the image center The effective focal length of the lens-camera assembly The radial lens distortion coefficient The pose (position and orientation) of the camera Parameters a) through d) are classified as intrinsic, e) as extrinsic. Memorial University of Newfoundland 3/15/2018

A systematic approach to the calibration of machine vision systems for industrial metrology Memorial University of Newfoundland 3/15/2018

A systematic approach to the calibration of machine vision systems for industrial metrology Memorial University of Newfoundland 3/15/2018

An Efficient Camera Calibration Technique Offering Robustness and Accuracy Over a Wide Range of Lens Distortion Memorial University of Newfoundland 3/15/2018

An Efficient Camera Calibration Technique Offering Robustness and Accuracy Over a Wide Range of Lens Distortion CALIBRATION ACCURACY AND THE LENS DISTORTION MODEL Memorial University of Newfoundland 3/15/2018

Underwater 3 D Vision Systems for MI Flume Tank Memorial University of Newfoundland 3/15/2018

Prototype Underwater Stereo Vision System Memorial University of Newfoundland 3/15/2018

Calibration Target for Underwater Stereo Vision System Memorial University of Newfoundland 3/15/2018

Development of an Intelligent 3 D Vision System for Underwater Environment through Actively Manipulated Laser Triangulation Memorial University of Newfoundland 3/15/2018

Development of an Intelligent 3 D Vision System for Underwater Environment through Actively Manipulated Laser Triangulation Memorial University of Newfoundland 3/15/2018

Industrial Machine Vision Applications Memorial University of Newfoundland 3/15/2018

Image Analysis Tools for Automated Inspection in the Food Processing Industry: X-Ray Enhancement Memorial University of Newfoundland 3/15/2018

Image Analysis Tools for Automated Inspection in the Food Processing Industry: Multispectral Imaging Memorial University of Newfoundland 3/15/2018

Image Analysis Tools for Automated Inspection in the Food Processing Industry: Multispectral Imaging Memorial University of Newfoundland 3/15/2018

High Speed, Line Scan Based Inspection System Memorial University of Newfoundland 3/15/2018

Line Scan Based Inspection System: Specifications • Objective: To remove defects (I. e. visible, dark particle larger than 0. 007”) from apple sauce • System must be able to handle 12 metric tons per 8 hour shift • System must remove 95% of visible defects Memorial University of Newfoundland 3/15/2018

High Speed, Line Scan Based Inspection System • 2 distinct challenges: • Detection • Removal Memorial University of Newfoundland 3/15/2018

High Speed, Line Scan Based Inspection System Memorial University of Newfoundland 3/15/2018

High Speed, Line Scan Based Inspection System Memorial University of Newfoundland 3/15/2018

Line Scan Based Detection System • Detection system based on a high performance line scan camera; 4096 pixels per line at 4800 lines per second. • Image acquisition and processing functions implemented on a Complex Programmable Logic Device (CPLD) as opposed to a microprocessor or Digital Signal Processor (DSP). • The objective is to implement image processing functions with dedicated logic gates (i. e. hardware) for real-time performance. Memorial University of Newfoundland 3/15/2018

What are CPLDs? • Complex Programmable Logic Devices (CPLDs) are a class of programmable logic device that are commonly used to implement complex digital designs on a single integrated circuit. • Applications of CPLDs in the field of computer engineering include the implementation of bus controllers, address decoders, priority encoder and state machines Memorial University of Newfoundland 3/15/2018

Line Scan Camera-Based Detection System Memorial University of Newfoundland 3/15/2018

Line Scan Camera-Based Detection System Memorial University of Newfoundland 3/15/2018

Line Scan Camera-Based Detection System Typical section of apple sauce recorded with an area scan camera Typical Particle Memorial University of Newfoundland 3/15/2018

Line Scan Camera-Based Detection System Memorial University of Newfoundland 3/15/2018

Line Scan Camera-Based Detection System Memorial University of Newfoundland 3/15/2018

Removal System Memorial University of Newfoundland 3/15/2018

Removal System Memorial University of Newfoundland 3/15/2018

Removal System: Flow Characterization • Rheological nomenclature and associated velocity profiles for steady flow through tubes with circular cross section. Memorial University of Newfoundland 3/15/2018

Viscosity Measurement Memorial University of Newfoundland 3/15/2018

Viscosity Measurement Velocity profile can be characterized by power law! Memorial University of Newfoundland 3/15/2018

Flow Profile of Apple Sauce Memorial University of Newfoundland 3/15/2018

Particle Removal Window Memorial University of Newfoundland 3/15/2018

System Timing Diagram Memorial University of Newfoundland 3/15/2018

Aspiration Valve Characterization Memorial University of Newfoundland 3/15/2018

Aspiration Valve Characterization Memorial University of Newfoundland 3/15/2018

Particle Detection Rate Versus Flowrate Memorial University of Newfoundland 3/15/2018

High Speed, Line Scan Based Inspection System: Current Status • Industrial partner is currently developing the production version of the system. • Packaging of the principle components (i. e. , lenses, cameras, electronics, light sources) remains a major challenge given the environment. • One possible solution is to integrate all the electronics in the camera enclosure. • Partner is anxious to explore applications in the pulp and paper industry. Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules • Objective: to automate the inspection of LCD modules in order to improve quality control • One step in the implementation of a Six-Sigma Program (“ 3. 4 defects per million opportunities”) • The inspection must be completed within 30 seconds for 10 predetermined LCD patterns • System can “learn” new LCD modules without modifying software Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of LCD modules: System Components • Pulnix camera with macro lens • High frequency fluorescent light sources • Coreco Bandit integrated image acquisition and VGA accelerator • Software developed using with Wi. T graphical programming environment in combination with Microsoft VB Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Original Image Showing Error in Alignment Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Thresholding Operation – Image Subtraction with respect to an image with no segments illuminated Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Blob Analysis – Reference Points are Identified Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Image Rotation and Translation Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Pixel by Pixel Image Subtraction from Reference Image – Thinning Operator Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules Blob Analysis Memorial University of Newfoundland 3/15/2018

Vision Based Inspection of Liquid Crystal Display (LCD) modules: Conclusions • Inspection system was installed at BAE Systems Canada Ltd. where it was used to test between 200 and 600 LCD displays per day. • Number of defective modules that passed inspection was basically reduced to zero. • Occasional “false positives” proved to be technical problems with the devices that previously went unreported. • Applications for this technology are numerous given the number of LCD displays produced annually. Blob Analysis Memorial University of Newfoundland 3/15/2018

QUESTIONS ? Memorial University of Newfoundland 3/15/2018