Скачать презентацию Computers and Photographs 1 Image Processing 2 Computer Скачать презентацию Computers and Photographs 1 Image Processing 2 Computer

462b97624ee0c72d98d0ee6ebf1c537e.ppt

  • Количество слайдов: 63

Computers and Photographs 1) Image Processing 2) Computer Vision Henry Schneiderman Computers and Photographs 1) Image Processing 2) Computer Vision Henry Schneiderman

Outline • Digital Cameras • Emerging Technology • Research in Image Processing and Computer Outline • Digital Cameras • Emerging Technology • Research in Image Processing and Computer Vision • Automatically Finding Faces and Cars in Photographs

Digital Cameras = Convenience • • Easy to capture photos Easy to store and Digital Cameras = Convenience • • Easy to capture photos Easy to store and organize photos Easy to duplicate photos Easy to edit photos

Digital Camera Usage • Lyra research report, 1999 Exposures in billions Digital Camera Usage • Lyra research report, 1999 Exposures in billions

State of the Art: Digital Cameras • Film is currently better in resolution and State of the Art: Digital Cameras • Film is currently better in resolution and color. – Professional photographers • Digital for low quality newspaper adds • Film for portrait photos • Computer storage limitations: 1 high resolution digital image = 25 Mega. Bytes • Printing –home printers not comparable to commercial printers

Future of Digital Cameras • Improved resolution and color • “Smart” cameras • More Future of Digital Cameras • Improved resolution and color • “Smart” cameras • More programmable features – Auto-focus on object of interest – “Everything in focus” photo – Capture photo when event X occurs

Photographs: Migration to Digital Format • Others means of digitizing imagery – Scanners (photo Photographs: Migration to Digital Format • Others means of digitizing imagery – Scanners (photo and film) – Frame-grabber for video

Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on computers 4. Other: manufacturing, military, games, . . .

Optical Character Recognition (OCR) • • First patent in OCR in 19 th century Optical Character Recognition (OCR) • • First patent in OCR in 19 th century First applications in post-office and banks State of the art not perfect. Examples of errors:

Handwriting Recognition • Works if constraints on writer, e. g. palm pilot Handwriting Recognition • Works if constraints on writer, e. g. palm pilot

Other document processing • Not just for text. . . • Examples: – Engineering Other document processing • Not just for text. . . • Examples: – Engineering document to CAD file – Maps to GIS format – Music score to MIDI representation

Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on computers 4. Other: manufacturing, military, games, etc

Biometrics • Technology for identification – Finger/palm print – Iris – Face Biometrics • Technology for identification – Finger/palm print – Iris – Face

Fingerprints • Minutae – spits and merges of ridges Fingerprints • Minutae – spits and merges of ridges

Face Identification • Not quite reliable yet. – Performance degrades rapidly with uncontrolled lighting, Face Identification • Not quite reliable yet. – Performance degrades rapidly with uncontrolled lighting, facial expression, and size of database • Several companies exist: – – – Visionics (Rockfeller Univesity spin-off) Eye. Matic (USC spin-off) Miros (MIT spin-off) Banque-Tec Intl (Australia) C-VIS Computer Vision (Germany) LAU Technologies • Commercial systems installed in London and Brazil to catch criminals

Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on computers 4. Other: manufacturing, military, games, etc

Management of images on computers • Compression – reducing storage size needed for images Management of images on computers • Compression – reducing storage size needed for images • Watermarking – Protecting copyright • Microsoft, Bell Labs, NEC, etc. Visible watermark

Photo-manipulation • Adobe Photoshop, Corel Photo. Paint, Pixami, Photo. IQ, etc. – More automatic Photo-manipulation • Adobe Photoshop, Corel Photo. Paint, Pixami, Photo. IQ, etc. – More automatic features

Searching Digital Image Collections • Large collections of images exist – Corbis 67 million Searching Digital Image Collections • Large collections of images exist – Corbis 67 million images – Getty 70 million stock photography images – AP collects 1000 s of digitized images per day • Search methods are inadequate – Rely on captions and colors • IBM’s Query by Image Content (QBIC) system

Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on Existing and Emerging Technology 1. Document scanning 2. Biometrics 3. Management of images on computers 4. Other: manufacturing, military, games, etc

Inspection for Manufacturing • Occum – inspection of printed circuit boards ($100 M / Inspection for Manufacturing • Occum – inspection of printed circuit boards ($100 M / year) • Cognex – Do-it-yourself toolkits for inspection (400 employees)

Automatic Target Recognition (ATR) • Finding mines, tanks, etc. • Billion dollar a year Automatic Target Recognition (ATR) • Finding mines, tanks, etc. • Billion dollar a year industry – Martin-Lockheed, TSR, Northrup-Grumman, other aerospace contractors. • Various types of imagery: – Synthetic Aperture Radar (SAR), Sonar, hyperspectral imagery (more than 3 colors)

Aerial Photo Interpretation / Automated Cartography • Classification of land-use: forest, vegetation, water • Aerial Photo Interpretation / Automated Cartography • Classification of land-use: forest, vegetation, water • Identification of man-made objects: buildings, roads, etc

Better Security Cameras • Cameras that are responsive to the environment – Track and Better Security Cameras • Cameras that are responsive to the environment – Track and zoom on moving objects – Automatic adjustment of contrast

Human-Computer Interaction • Computer games that involve interaction with user • Intelligent teleconferencing Human-Computer Interaction • Computer games that involve interaction with user • Intelligent teleconferencing

Medical imagery • Medical image libraries for study and diagnosis • Image overlay to Medical imagery • Medical image libraries for study and diagnosis • Image overlay to guide surgeons

History • 1980’s ~100 companies – manufacturing applications mostly • Early 1990’s less than History • 1980’s ~100 companies – manufacturing applications mostly • Early 1990’s less than 10 companies • Late 1990’s ~100 companies – face recognition, intelligent teleconferencing, inspection, digital libraries

Computer Vision and Image Processing Research Computer Vision and Image Processing Research

• Image processing image • Computer vision chair, face, shape, etc. image “Symbolic” • Image processing image • Computer vision chair, face, shape, etc. image “Symbolic” description

1. Image Processing: Filtering 1. Image Processing: Filtering

2. Image Processing: Compression • Lossless – No loss in quality, gif, tiff • 2. Image Processing: Compression • Lossless – No loss in quality, gif, tiff • Lossy – Original image cannot be reconstructed, jpeg

3. Image Processing: Watermarking • Information hiding – Protecting Copyright 3. Image Processing: Watermarking • Information hiding – Protecting Copyright

4. Image Processing: Transformation • Transforming image can make it easier to analyze Wavelet 4. Image Processing: Transformation • Transforming image can make it easier to analyze Wavelet transform of image

Decomposition in Resolution/Frequency coarse intermediate fine Decomposition in Resolution/Frequency coarse intermediate fine

Wavelet Decomposition Vertical subbands (LH) Wavelet Decomposition Vertical subbands (LH)

Wavelet Decomposition Horizontal subbands (HL) Wavelet Decomposition Horizontal subbands (HL)

1. Computer Vision: 3 D Shape Reconstruction • Use images to build 3 D 1. Computer Vision: 3 D Shape Reconstruction • Use images to build 3 D model of object or site 3 D site model built from laser range scans collected by CMU autonomous helicopter

2. Computer Vision: To guide Motion • Visually guided locomotion – robotic vehicles • 2. Computer Vision: To guide Motion • Visually guided locomotion – robotic vehicles • Visually guided manipulation – Hand-eye coordination CMU Nav. Lab II

3. Computer Vision: Recognition and Classification 3. Computer Vision: Recognition and Classification

Challenges in Object Recognition 245 267 234 142 22 28 38 121 156 187 Challenges in Object Recognition 245 267 234 142 22 28 38 121 156 187 98 73 32 12 123 21 21 38 209 237 121 99 87 59 197 216 244

Challenges in Object Detection • Intra-class variation Challenges in Object Detection • Intra-class variation

• Lighting variation • Lighting variation

• Geometric variation • Geometric variation

Simpler Problem: Classification • Fixed size input • Fixed object size, orientation, and alignment Simpler Problem: Classification • Fixed size input • Fixed object size, orientation, and alignment “Object is present” (at fixed size and alignment) Decision “Object is NOT present” (at fixed size and alignment)

1) Apply Local Operators f 1(0, 0) = #5710 f 1(0, 1) = #3214 1) Apply Local Operators f 1(0, 0) = #5710 f 1(0, 1) = #3214 fk(n, m) = #723

2) Look-Up Probabilities f 1(0, 0) = #5710 P 1( #5710, 0, 0 | 2) Look-Up Probabilities f 1(0, 0) = #5710 P 1( #5710, 0, 0 | obj) = 0. 53 P 1( #5710, 0, 0 | non-obj) = 0. 56 f 1(0, 1) = #3214 P 1( #3214, 0, 1 | obj) = 0. 57 P 1( #3214, 0, 1 | non-obj) = 0. 48 Pk( #723, n, m | obj) = 0. 83 fk(n, m) = #723 Pk( #723, n, m | non-obj) = 0. 19

Probabilities Estimated Off-Line f 1(0, 0) = #567 H 1(#567, 0, 0) = H Probabilities Estimated Off-Line f 1(0, 0) = #567 H 1(#567, 0, 0) = H 1(567, 0, 0) + 1 H 1(#567, 0, 0) P 1(#567, 0, 0) = fk(n, m) = #350 S H 1(#i, 0, 0) Hk(#350, 0, 0) = Hk(#350, 0, 0) + 1 Hk(#350, 0, 0) Pk(#350, 0, 0) = S Hk(#i, 0, 0)

3) Make Decision P 1( #5710, 0, 0 | obj) = 0. 53 P 3) Make Decision P 1( #5710, 0, 0 | obj) = 0. 53 P 1( #5710, 0, 0 | non-obj) = 0. 56 P 1( #3214, 0, 1 | obj) = 0. 57 P 1( #3214, 0, 1 | non-obj) = 0. 48 Pk( #723, n, m | obj) = 0. 83 Pk( #723, n, m | non-obj) = 0. 19 0. 53 * 0. 57 *. . . * 0. 83 0. 56 * 0. 48 *. . . * 0. 19 >l

Overall Algorithm f 1(0, 0) = #5710 P 1( #5710, 0, 0 | obj) Overall Algorithm f 1(0, 0) = #5710 P 1( #5710, 0, 0 | obj) = 0. 53 P 1( #5710, 0, 0 | non-obj) = 0. 56 f 1(0, 1) = #3214 P 1( #3214, 0, 1 | obj) = 0. 57 P 1( #3214, 0, 1 | non-obj) = 0. 48 0. 53 * 0. 57 *. . . * 0. 83 0. 56 * 0. 48 * … * 0. 19 fk(n, m) = #723 Pk( #723, n, m | obj) = 0. 83 Pk( #723, n, m | non-obj) = 0. 19

Detection: Apply Classifier Exhaustively Search in position Search in scale Detection: Apply Classifier Exhaustively Search in position Search in scale

View-based Classifiers Face Classifier #1 Face Classifier #2 Face Classifier #3 View-based Classifiers Face Classifier #1 Face Classifier #2 Face Classifier #3

• 2 classifiers trained for faces. • 8 classifiers trained for cars. • 2 classifiers trained for faces. • 8 classifiers trained for cars.

Training Classifiers • Cars: 300 -500 images per viewpoint • Faces: 2, 000 images Training Classifiers • Cars: 300 -500 images per viewpoint • Faces: 2, 000 images per viewpoint • ~1, 000 synthetic variations of each original image – background scenery, orientation, position, frequency • 2000 non-object images – Samples selected by bootstrapping • Minimization of classification error on training set – Ada. Boost algorithm (Freund & Shapire ‘ 97, Shapire & Singer ‘ 99) • Iterative method • Determines weights for samples

Applications of Face Detection • Automatic red-eye removal from photographs • Automatic color balancing Applications of Face Detection • Automatic red-eye removal from photographs • Automatic color balancing in photofinishing • Intelligent teleconferencing • Component in face identification system

Difficulty Increases with Complexity of Object • • 2 D vs. 3 D Specific Difficulty Increases with Complexity of Object • • 2 D vs. 3 D Specific objects – e. g. my coffee mug A category of objects – e. g. all coffee mugs Amount of intra-category variation – Rigid or semi-rigid structure, e. g. face – Articulated objects, e. g. human body – Functionally defined objects, e. g. chairs

Summary: Image Processing & Computer Vision • Not as mature as speech recognition – Summary: Image Processing & Computer Vision • Not as mature as speech recognition – Technology not as reliable – Fewer companies, fewer products • Success on limited problems, e. g. , documents • More applicable to fault tolerant problems • Technology will grow – Emergence of digital camera – Improved methods