Task 3 Semi-Automatic System for Pollen Recognition

Task 3 : Semi-Automatic System for Pollen Recognition Partners: – REA (Barcelona) – REA (Cordoba) – LASMEA (Clermont-Ferrand) – INRIA (Sophia-Antipolis) 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Global architecture schema 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Material and Methods • System Hardware: – – – Light microscope 3 axes micro-positionning device CCD colour camera Image acquisition card PC computer • Pollen slides are prepared by technicians – Pollen grains are sampled using Hirst traps – Pollen grains are coloured with fuchsine (4µg/100 ml) 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

System Hardware A light microscope is driven automatically by a computer 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

3 mm 13 mm slide Adhesive (from hirst sampler) label 25 mm (slide width) Slide preparation for automatic analysis 75 mm (slide length) • Choice of final coloration: 4µg/100 ml 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Man-Machine Interface The system can work in supervised or automatic mode 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Man-Machine Interface • Sweeping scheme acquisition and convertion • Slide information acquisition (for slide information automated management) 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Pollen detection and extraction • Sweep and scan the slide for pollen grains (x, y) • Focus on the objects of the scene (z) – Optimisation of a sharpness criterion (SML) • Detect pollen grains in the scene (2 D) – Colour coding: principal component analysis on RGB – Split-and-merge segmentation with markovian relaxation – Pollen detection by colour analysis (luminance/chrominance) • Extract pollen grains (3 D) – 3 D digitisation from the central image 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Pollen Grain Detection and Localisation • Automated sweeping and localisation (x 20 or x 40 lens) • Fully automated or semi-automated extraction of pollen grains (x 40 or x 60 lens) Segmentation Localisation Automatic pollen grain detection 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Pollen Grain Extraction • 3 D acquisition of pollen grains – set of images at different depths For each grain • 100 optical sections • step = 0. 5 microns Features may appear on different heights 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Validation of the image acquisition module Validation with aerobiological slides: Btu 1101 d 2 Btu 2100 d 3 Example of good results Example of bad results • True positive detection (pollen detected) : 80% • False positive detection (non-pollen detected) : 25% 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Global architecture schema 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Goal for recognition • Identify the 4 ASTHMA types from other pollen types • Similar in appearance • Flowering at the same time of the year Detect true positive • Do not identify another type as an ASTHMA type Avoid false positive 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Main Pollen Types Studied and Similars Cupressaceae Olea Parietaria Poaceae Populus Brassicaceae Fraxinus Ligustrum Phillyrea Salix Broussonetia Morus Urtica membranacea Celtis Coriaria 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Reference / Aerobiological images Reference images: Aerobiological images: 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Knowledge acquisition General knowledge acquisition Specific knowledge acquisition 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Palynological Knowledge • The system tries to mimic the palynologists • Pollen knowledge is used to identify each grain • Knowledge sources from – Palynology – Aerobiology 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Second module: Pollen recognition • First step: coarse classification – Global measures on the grain (2 D) • Size, colour (RGB), shape, convexity, . . . – Sampling date (external data for flowering season) – First estimations of possible types (sorted hypothesys list) 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Global measures & flowering information 1) Global measures – taken on the central image of the grain – colour, size, shape, … 2) Flowering information – taken from pollinic calendar 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Second module: Pollen recognition • Second step: fine classification – Search specific characteristics (3 D) – Need specific knowledge about pollen types – Driven by the first hypotheses (test only most possible types) – Refine until no ambiguity remains 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Search pollen characteristics – 3 D analysis is made to find possible locations – Search is done on isolated pollen masks Focus analysis vs Image number Image Full Grain Interior Exine 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Search of specific characteristics Example: Poaceae pore Image 35 /100 02/10/2001 Image 50 / 100 Image 65 / 100 Task 3: Semi-automatic System for Pollen Recognition Image 80 / 100

Search of specific characteristics Example: Cupressaceae cytoplasm Above central image Below central image Sum of bright regions 02/10/2001 Sum of dark regions Task 3: Semi-automatic System for Pollen Recognition

Search of specific characteristics Example: Olea reticulum – Network located on the external surface of the grain – Detection steps: • Check if the grain is reticulated • Localise the reticulum (3 D) • Analyse the reticulum 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Aerobiological images: dust removal 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Validation of the recognition module • Problems with reference validation data sets: Development set 02/10/2001 Validation set 2 Task 3: Semi-automatic System for Pollen Recognition

Validation of the recognition module • Validation using leave-one-out method – Testing using the development set • Learn parameters on all but one pollen grain • Test one pollen grain • Loop for each pollen grain – Simulation of the sampling date (flowering - Bellaterra) • Results: – Cupressaceae: 94. 4% – Olea: 100% - Parietaria: 100% - Poaceae: 100% – 5 classes (ASTHMA types / others): 99. 7 % – 32 classes (all pollen types): 77. 7 % 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Validation of the semi-automatic system for pollen recognition • Acquisition module – Excellent results, except for Olea – Interface should be improved for non-experts in computer science like palynologists • Recognition module – Shows the feasibility of pollen recognition – Further investigations are needed to be more robust 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition

Validation of the semi-automatic system for pollen recognition • Prospects: – Calibration • automatic procedure to ensure standard conditions – Data set (training) not representative enough • reference / aerobiological • need complete data sets for all pollen types (not only 4 types) – Algorithms are too much colour-dependant • find and develop new algorithms more robust 02/10/2001 Task 3: Semi-automatic System for Pollen Recognition