LIBRARIES AND ALGORITHMS: Automated Interpretation of VIS-SWIR Data and the Advantage of Special Purpose Reference Databases Phoebe L. Hauff Cari Deyell-Wurst William Kerby PRIZE
INTRODUCTION “Spectral geology” applications (including satellite, airborne, core scanning and field measurements) have become very common due to significant technological advancements and improved instrumentation. This has also led to many programs that attempt to automate the sample handling and mineral identification processes. There are numerous libraries and algorithms available. They are briefly described here. We have also done a Round Robin to demonstrate effectiveness of some of the algorithms. Expanded summaries (www. spectral-international. com)
OVERVIEW • REVIEW OF SPECTRAL LIBARIES • REVIEW OF ID ALGORITHMS • INTERACTION OF BOTH • ROUND ROBIN RESULTS
SPECTRAL LIBRARIES Data Bases contain spectra, ancillary information, physical • Spectral libraries contain reference spectra, which are properties, references, associated species, location compared against an unknown spectrum using computer automated ID techniques. • Without them, it would be difficult to do interpretation Minerals are highly variable – composition, wavelength, profile, crystallinity Difficult to automate CRYSTALLINITY CHLORITES
KNOWN VIS-SWIR LIBRARIES The libraries listed here are the better known ones. There are innumerable little ones targeted at one mineral, one mineral group, vegetation. The addresses for the common ones are included and will be on the SII website Spec. MIN: SPECMIN is a mineral identification system for spectroscopy that includes an extensive and dynamic library of reference spectra for minerals, wavelength search/match tables, physical properties of each species in the database, and literature references for the infrared active mineral phases. The spectral library includes a minimum of two different samples per mineral that show compositional differences within mineral species. In addition to mining applications, SPECMIN can also be used in remote sensing applications for ground truthing. www. spectral-international. com truthing. CSIRO: Inbuilt reference library of spectra of common minerals. As well as the mineral spectra, the library also includes some artifact materials such as vegetation, plastic and marker pen which could also potentially contribute to your project spectra. www. csiro potentially contribute to your project spectra. USGS: The concept and identification work basic to this library was started by Dr Graham Hunt in the 1970’s. The library is used as a reference for materials identification in remote sensing images. It can also be used for identification of laboratory and field spectrometer data. The software used to manage this library is spec. PR. The library is available without cost as a download from the internet. http: //speclab. cr. usgs. gov/spectral. lib 06
KNOWN VIS-SWIR LIBRARIES Brown University RELAB The public domain spectral library is supported by NASA at Brown University. WWW. PLANETARY. BROWN. EDU/RELAB/. It contains thousands of spectra from NASA researchers and others. Data cannot be used for commercial applications. It contains many project specific data sets including planetary projects. Arizona State University MINESpectra is a data management program that interfaces with the USGS, JHU and special purpose libraries. It is free. www. geologynet. com/minspectra. htm Cal Tech VIS This extensive library concentrates on the VIS range. There does not appear to be an identification program. It does not say if this is digital or not. Minerals. gps. caltech. eduindex. html Johns Hopkins FTIR , SWIR Incorporated by JPL into ASTER library. Available from JPL
KNOWN VIS-SWIR LIBRARIES JPL The spectral library available from the Jet Propulsion laboratory contains 160 mineral spectra in 3 different grain sizes. The references are well characterized. This library was incorporated into the “ASTER” Library along with the Johns Hopkins midinfrared library growing to 2400 spectra of different infrared materials. It is available from JPL. Speclib. JPL. nasa. gov/documents/jpl_desc MINEO (www 2. brgm. fr/mineo/spectral. htm) All spectra collected during field campaigns, lab analysis and image analysis are gathered into a single spectral library. This MINEO specific spectral library will constitute a European scale spectral data base for mining related contaminated areas. It is able to manage large amounts of spectra collected from laboratory analysis, field spectrometry, as well as spectra extracted from hyperspectral imagery.
THE MINEO PROJECT∗ THE MINEO PROJECT Chevrel S. , BRGM, Orléans – France, Kuosmannen V. , GTK, Espoo – Finland; Belocky R. , GBA, Wien – Austria; Marsh S. , BGS, Nottingham, United Kingdom; Tukiainen T. , GEUS, Copenhagen – Denmark; Mollat H. , BGR, Hanover – Germany; Quental L. , IGM, Lisbon – Portugal; Vosen P. , DSK, Bottrop – Germany, Schumacher V. , JRC/SAI, Ispra – Italy, Kuronen E. , Mondo Minerals, Kajaani – Finland, and Aastrup P. , NERI, Copenhagen – Denmark ABSTRACT MINEO is a European Research and Technological Development project which aims at developing tools and methods for assessing and monitoring the environmental impact of mining activities by means of combined Earth Observation and other relevant environmental data set. MINEO is designed to improve the already proven hyperspectral imagery capabilities in mineral mapping for use in the mapping of miningrelated contaminated areas in European vegetated environments. MINE WASTE LIBRARY Generation of an European scale spectral library of contaminated areas
APPLICATIONS AMD
SPECIAL-PURPOSE LIBRARIES vs. ONE MAIN LIBRARY BY ALTERATION TYPE BY SPECIFIC DEPOSIT TYPE SITE- SPECIFIC USGS
EXISITING DEPOSIT-SPECIFIC LIBRARIES GOLD EPITHERMAL HSS, LSS GOLD OROGENIC, VEIN PORPHYRY IRON MINERALS IOCG URANIUM - UNCONFORMITY SKARNS REE
SPECIAL-PURPOSE LIBRARIES: ALTERATION TYPES ALTERATION PROPYLLITIC - ZEOLITIC ARGILLIC INTERMEDIATE ARGILLIC ADVANCED ARGILLIC SERICITIC-CHLORITE SILICIC TOURMALINE SKARNS QSP (QUARTZ-SERICITEPYRITE) CARBONATE ADVANCED ARGILLIC
ADVANTAGES OF SPECIAL PURPOSE LIBRARIES FEWER WRONG CHOICES = MORE ACCURATE MATCHES Example: Porphyry Deposit – Chile – Lithocap Zone 50 Random samples selected through mineralized zone Samples run against: SPECMIN Database – SII library 1536 reference samples Deposit Library – 557 samples from porphyry Environments + selected deposit references TSG (TSA) Database Mineral suite seen through zone, alunite, dickite, kaolinite, smectite, illite, muscovite, chlorite, epidote, gypsum, silicification, goethite, hemitite, jarosite
Procedure: 1 - Samples run against large SII library 2 – Samples run against a location-specific porphyry library Results: SII vs. Porphyry library Results between SII vs. porphyry database identified 14% of first order minerals were misidentified with SII Main library Minerals causing largest issue were smectite-illite-muscovite presence.
Results: Porphyry Library vs. TSG 22% of first order minerals misidentified with TSG 5% of first order minerals should have been identified as second order presence 34% of second order minerals were identified as NULL in TSG (mineral presence did exist in 90%) 37. 5% of remaining second order minerals were misidentified Only 23% of second order minerals correctly identified
Example: Porphyry Cu-Au Deposit Mineral ID Using General Library Spec. MIN - Feature. Search Top matches not
Example: Porphyry Cu-Au Deposit Mineral ID Using Deposit-Specific Library Spec. MIN - Feature. Search Top matches correct!
Example: Porphyry Cu-Au Deposit Mineral ID Using General Library Spec. MIN - Feature. Search Top matches not
Example: Porphyry Cu-Au Deposit Mineral ID Using Deposit-Specific Library Spec. MIN - Feature. Search Top matches correct!
SPECIAL PURPOSE LIBRARIES WILL PROVIDE BETTER MATCHING STATISTICS THEY ELIMINATE WRONG CHOICES
“AUTOMATED” MINERAL ID ALGORITHMS The Spectral Geologist (CSIRO, Australia) The Corescan Interpretation Software (Core. Scan) Feature. Search (Steve Mackin, spec. MIN) Spec. PR (USGS) - NOT ID PROGRAM The original hyperspectral ID program - NOT AN ID PROGRAM (Neil Pendock, Phil Harris, Paul Linton, Anglo American-De. Beers) Newmont – Dave Coulter - NO LONGER USED Rio Tinto - Alistair LAMB - NO LONGER USED MINEO – French Consortium - PROJECT ENDED BHP - PROJECT STOPPED WHEN PIMA APPEARED
“AUTOMATED” MINERAL ID ALGORITHMS: TYPES of PROGRAMS Wavelength-based Least squares Profile based Linear regression Neural nets Non-linear regression
“AUTOMATED” MINERAL ID ALGORITHMS: TYPES of PROGRAMS SHAPE BASED – Feature Position simple lookup table example: Feature Search; Tetracorder (USGS) SHAPE MATCHING Pearson correlation Matrix Matched filtering VECTOR SPACE ALGORITHMS remote sensing classification method TSG/TSA appears to use this type of algorithm
The Spectral Geologist (CSIRO, Australia) Specialist processing and analysis software package designed for analysis of field or laboratory spectrometer data. It is automated It uses a spectral library developed by CSIRO
Corescan is a global services company specialising in the The Corescan hyperspectral scanning, Interpretation Software processing and analysis of drill core, rock chips and other (Core. Scan) geological samples for the mining, oil and gas, and geotechnical industries
Feature. Search (Steve Mackin, spec. MIN) Feature. Search is a semi-automatic mineral identification package for determining mineralogy based on features observed in an "unknown" spectrum collected by a field spectrometer. Ideal for novice users with little experience in spectral identification or for advanced users trying to determine low proportion end-members in mixtures. The software is spectrometer-independent and operates with data from spec. TERRA™, ASD, GER, SEI or PIMA spectrometers. The users can select a spectral library created with any spectrometer. Drag and drop a file or select a file from Plot Preview, click on the "Search Library" button and the chosen mineral library is searched in less than a second. The results are displayed clearly to allow the user to extract and save the information of interest in History Libraries. Use the extracted end-members to build a deposit or environment specific library to use in an automatic mineral identification algorithm such as SIMIS Field 2. 9, or save the results to directly import into Microsoft Excel.
SPECMIN incorporates options from the Feature. Search program, which allows it to unmix spectral components, do mineral percentages, and access user-created custom libraries. SPECMIN is a data management system that puts spectral data into an easy access format. It provides numerous spectral libraries including ASD, PIMA, USGS, and JPL mineral libraries. It contains spectra from hyperspectral imagery such as AVIRIS and SFSI. SPECMIN also contains libraries for soils, and libraries specific to precious metals deposit types.
Example: Mineral Spectral Analysis Software Potential to change mineral list? Spectra Kaolinite Illite Dolomite Dickite Phlogopite Epidote Calcite Alunite Pyrophyllite Chlorite sample 1 61. 3 11. 4 0. 1 13 0. 1 13. 1 0. 5 0. 1 sample 2 39. 7 59. 3 0. 1 0. 4 0. 1 sample 3 41. 4 21. 9 0. 1 3. 4 0. 1 16. 5 0. 1 16. 5 sample 4 3. 9 1 17. 7 0. 1 30. 3 3. 4 0. 1 43. 2 sample 5 25. 1 22. 6 6 0. 1 3. 2 7. 3 7. 8 0. 1 27. 6 sample 6 0. 2 98. 2 0. 2 sample 7 38. 4 16. 5 0. 1 5. 6 0. 1 11. 4 0. 1 27. 7 Potential to be effective in localized environments! Total Sums to 100%
WHY SO FEW? ? ? IT IS HARD
CHALLENGES FOR AUTOMATED MINERAL ID PROGRAMS CHEMICAL VARIABILITY OF MINERALS ABSORPTION CO-EFFICIENTS ARE UNKNOWN NON-LINEAR ASSOCIATIONS MIXTURES
Ex. Absorption Coefficients In A high (illite) and low (chlorite) reflectance mixture, the low reflector is difficult to see. In this example, there has to be nearly 40% Fe-chlorite present before it can be detected Fe-Chlorite & illte
OBJECTIVES: ROUND ROBIN TEST STUDY SURVEY OF AUTOMATED MINERAL ID PROGRAMS • S: N. • Quality of spectrum accepted • What the algorithms are prejudiced towards • Where they do not do well • Poor libraries? • Special purpose data libraries • Artifacts of the spectrum • Mixtures • Experienced user required? Or “out of the box”?
ROUND ROBIN: SUMMARY OF PROCEDURE • 48 spectra total (natural samples, computergenerated mineral mixtures) • spec. TERRA, Terra. Spec, Field. Spec Pro • Anonymous participants • Samples were run through automated mineral ID programs of participants’ choice • The entries were scored • A winner determined based on: • Greatest number of correct minerals identified • Penalized for wrong answers
MINERALS IN THE STUDY
MINERALS IN THE ROUND ROBIN KEY Kaolinite Dickite Alunite pyrophyllite Diaspore Topaz Dumortierite Xxxxxxxxxxxxx wm Illite Smectite Mixed Layer I/S Saponite Scapolite Silica Dolomite Calcite Chlorite Amphibole Pyroxene Tourmaline Biotite Diopside Apophylite Prehnite Pumpellyite REE Hematite Goethite
ROUND ROBIN SPECTRA: EXAMPLES Szmolnokite Elbiate + Lepidolite Monazite Dickite + alunite + pyrophyllite
ROUND ROBIN: MINERAL ID PROGRAMS ENTERED • GRAMS • TSG (several versions) • FEATURE SEARCH • TNT-MIPS • IN-HOUSE C+ • MSA (MINERAL SPECTRAL ANALYSIS)
Summary of Programs Submitted to Round Robin: Strengths and Weaknesses Program Strengths The Spectral Geologist Up to 5 minerals (TSG) – TSA (The potentially identified Spectral Assistant) Weaknesses Minor components of mixtures poorly identified TNT MIPS • Good attempt at Second and third order mixtures – up to 5 phases of mixtures not phases well identified • Fe-phases identified MSA – Mineral Spectral Analyst Mineral ‘matrix’ could work very well for major components No measure of reliability – will always fit spectra to given minerals “In-house” C Attempt to determine VNIR phases Matches only 1 spectra
ROUND ROBIN: Participants and winner Participant # Program Version 8011 TNT MIPS PRO-3. 1 8014 TSG 7. 1 8015 GRAMS 8016 -1 FS 8017 TSG 8018 in-house C 8021 TSG 7. 1 3. 0 8022 MSA Mineral Spectral Analysis V 3. 6 8024 TSG 5. 03 8025 TSG 7. 1 8030 8035 Data Base Country SPECMIN Turkey TSG Australia Specmin USA SPECMIN Argentina unspecified USA Specmin ENVi format USA Australia China IN TSG Netherlands NG TSG? Australia TSG 2 answers in TSG Chile proprietary 1 specmin USA
OBSERVATIONS & COMMENTS BIGGEST OBSERVATION not a lot of progress made over the last 10 years TSG CAN GENERATE HIGHLY VARIABLE RESULTS Specialized libraries definitely improve matching statistics Library must be comprehensive to provide accurate answers The less complicated the procedure, the more accurate will be the results Less choices, better results – i. e. special purpose libraries provide the best answers
ROUND ROBIN: RESULTS Computer program: The Spectral Geologist v. 5. 03 Database: The Spectral Assistant (TSA) Dr. F. J. A. (Frank) van Ruitenbeek University of Twente, Netherlands PRIZE
IF USER DOES NOT KNOW WHAT ANSWERS ARE WRONG, HOW CAN ANSWERS FROM AUTOMATED PROGRAMS BE EVALUATED? ?
Скачать презентацию LIBRARIES AND ALGORITHMS Automated Interpretation of VIS-SWIR Data
937c21513b13db2d78244fcb7a578b10.ppt