USGS QA Plan for Digital Aerial Cameras l

USGS QA Plan for Digital Aerial Cameras l l l Briefing to Headquarters August 2007 USGS Remote Sensing Technologies Project http: //calval. cr. usgs. gov/ Greg Stensaas, stensaas@usgs. gov George Lee, gylee@usgs. gov U. S. Department of the Interior U. S. Geological Survey

Remote Sensing Technologies Project Characterization Tasks l l l Satellite Characterization and Calibration u US system calibration and characterization group u Joint Agency Commercial Imagery Evaluation (JACIE) Team u Landsat Data Gap Assessment u Future of Land Imaging Aerial Mapping Sensor Characterization and Calibration u Film Camera Calibration u Digital Aerial System Product Characterization u Digital Camera Characterization Small/Medium Format Cameras u In situ Calibration for analog and digital sensors u USGS Quality Assurance Plan for Digital Aerial Imagery Range Characterization and Calibration Sites Ground Instrumentation Sites Assessment of Remote Sensing Technologies 2

History l USGS calibrating aerial film cameras since 1973 l USGS has established requirements and standards for photo imagery for many years u Film camera standards since 1950’s with the National Bureau of Standards National Map Accuracy standards since the 1940’s u de facto standard for film mapping cameras u l In 1999, ASPRS Panel recommended that USGS: u u u l 1) Continue and improve OSL film cameras calibration, 2) Develop camera calibration capability, 3) Conduct research efforts on digital acquisition systems, 4) Develop an in situ (flight) calibration process, 5) Establish satellite imagery independent cal/val process, and 6) Develop a U. S. Standard for camera and sensor calibration. New USGS project created u u Cross-center, cross-program teams actively working ASPRS recommendations USGS has developed and is implementing a USGS QA Plan for Digital Aerial Imagery 3

l Joint Agency Commercial Imagery Evaluation (JACIE) 6 th Annual Workshop held March 20 -22, 2007 u u l l USGS, NGA, USDA, and NASA Collaboration (USDA joined this year) NOAA looking at coming on board in FY 08 March 11 -13, 2008 workshop – contracting in FY 07 Workshop information @ http: //calval. cr. usgs. gov/jacie. php u Enhanced scope to Medium Resolution Satellite & Aerial Sensors useful to the remote sensing community – U. S. and International systems --- u Request for a short Lidar and SAR session l Independent assessment of product quality and usability l New applications and understanding of remotely sensed data 4

Digital Aerial Imagery QA l l l USGS Plan for Quality Assurance of Digital Aerial Imagery u Contracting Guidelines u Manufacturers Certification u Data Providers Certification u Image Quality Assurance and Assessment Guidelines Sponsors: u USGS Land Remote Sensing Program, USGS National Geospatial Program, USDA APFO, IADIWG The Plan: u Certifications will be used in contract evaluation process (beginning FY 08 timeframe) u Contracting and Image Assessment Guidelines (FY 07 -08) u Education of Federal, State, and Local Gov’t Contracting personnel (Started in FY 07) 5

IADIWG l IADIWG: Inter-Agency Digital Imaging Working Group l l Represents the 14 leading US Government agencies involved in aerial imaging l Cooperating agencies: BOR, BLM, COE, DOT, EPA, FSA, FWS, NASA, NIST, NGA, NOAA, NRCS, USFS, and USGS l l Formed and chaired by USGS in 2005 Monthly telecons and bi-yearly face-to-face meetings USGS Plan approved and supported by IADIWG u Govt. and industry support via workshops and conferences 6

Contracting Guidelines l User community is not sure how to contract for digital imagery u u u New terms, capabilities, standards, lexicon Inhibits digital contracting Addresses boilerplate requiring “USGS Certificate” Goal is to remove barriers to digital aerial contracts Encourage digital imaging! l Created Federal Digital Imagery General Contract Guideline l IADIWG continuing u IADIWG advised of need for data and interoperability guidelines for digital imagery including LIDAR 7

Digital Aerial Imagery Specification Tool 8

QA Plan Status l l l USGS Plan for Quality Assurance of Digital Aerial Imagery u Initial plan approved by IADIWG and USGS u Briefed to ASPRS, MAPPS, and User Workshop without major issues Manufacturers Certification Plan u Revised based on 4 visits, finalizing documents for future u New types being certified – DSS 322 & 439, ADS 40 -SH 52 Data Providers Certification Plan u In review by 5 Data Providers – complete plan and begin certification process this year u In-situ ranges being established across country u In-situ range specification document being completed u Have done initial characterization work for Data Providers systems 9

Digital Aerial Systems Certification l USGS certifications completed: u Applanix DSS l u Intergraph Zeiss DMC l u http: //www. vexcel. com/products/photogram/ultracam/ Leica ADS-40 l l http: //www. intergraph. com/dmc/ Microsoft Vexcel Ultra. Cam D (new Ultra. Cam X) l u http: //www. applanix. com/products/dss_index. php http: //gi. leica-geosystems. com/LGISub 1 x 2 x 0. aspx USGS certifications requests u Digital Modular Camera (Di. MAC) l u http: //www. dimacsystems. com/ Airborne Data Systems l http: //www. airbornedatasystems. com/ 10

Manufacturer Certification l l l Certifications provided on Dec 2006 for Applanix DSS, Intergraph DMC, and Vexcel Ultra. Cam-D Leica ADS-40 including SH-51/52 Proprietary report stored at USGS Public version being worked and will be available on web Others in the queue for FY 08 11

USGS Review Team l Review Team Lead - Gregory L. Stensaas Remote Sensing Technologies Project Manager USGS Earth Resources Observation and Science Center, Sioux Falls, SD stensaas@usgs. gov l Systems Engineering Team Member - Jon Christopherson Science Applications International Corporation (SAIC) Contractor to the USGS EROS, Sioux Falls, SD jonchris@usgs. gov l Photogrammetric Engineering Team Member - Dr. George Y. G. Lee U. S. Geological Survey, Menlo Park, CA gylee@usgs. gov l Geo-Spatial and Software Engineering Team Member - Donald Moe Science Applications International Corporation (SAIC) Contractor to the USGS EROS, Sioux Falls, SD dmoe@usgs. gov l Radiometric and Physics Team Member - Dr. Robert Ryan Science Systems and Applications, Inc. Contractor to NASA Stennis Space Center, MS Robert. Ryan@ssc. nasa. gov 12

Digital Aerial Data Providers Certification involves 2 components: u u l Quality process evaluation of data providers from system installation and checkout through data acquisition and delivery System and product assessment Focused on processes and process control u u Ensures that Data Providers are operating sensors in accordance with manufacturer’s instructions and limitations Ensures that Data Providers follow quality procedures 13

Data Providers Certification Team l USGS Dale Benson u Bryan Christensen u George Lee u Lin Neifert u Dana Shippy u Greg Stensaas u Tom Sturm u l USDA/APFO u John Mootz l Supported by many others 15

QA Plan Status (cont. ) l l l Contracting Guidelines and Specification tool u Initial documents complete and web tool in work u Plan to coordinate image visualization tools Training u Many workshops and conference presentations u Work with conference groups and state liaisons to set up workshops Quality Assessment u Develop digital imagery evaluation tool, documentation, and contract QA/QC guidelines u Update guidelines with QA/QC requirements u Enhance and develop statistical QA/QC methods with recommended contract publishable statistics baseline and contract performance guidelines 16

Imagery Assessment and QC team l l USGS u Daniels u Bryon Ellingson u Robert Kelly u Tom Sturm u Lin Neifert USDA/APFO u David Davis, FSA u Brenda Simpson, FSA QA Team Lead NOAA u Bill Stevenson Supported by many others 17

Digital Data Acceptance Standards l End-users unsure of how to judge digital aerial data quality u New terms & capabilities (resolution, spectral, etc. ) u Each customer understands things differently l There is a need for common, uniform definitions and methods for evaluating quality of image data l USGS to work with IADIWG and technical experts to develop standards and guidelines l Goal is a Web-based tool illustrating quality problems, measurement techniques, and standards u Working with NASA Stennis to upgrade imagery guideline 18

USGS Implementation Plan – Manufacturer Certification l Current Plan - - USGS LRS cost-sharing: u LRS keeps up with technology, maintains industry and system currency and associated relevance to science l Future plan - - LRS provides infrastructure manufacturer reimbursable costs of $20 K to $25 K per system; l FY 08 Cost - - $143, 000 after reimbursable 19

USGS Implementation Plan – Data Provider Certification l Current Plan - DP cert process review every 3 years or as system changes and in-situ evaluation once per year l Reimbursable cost per data provider – $10 K; Funding will be used to support team and in situ evaluations and range infrastructure l FY 08 Cost: u GIO covers the infrastructure cost for process evaluation and team leadership l u Assuming not more than 10 data providers per year, $153, 000 LRS provides infrastructure cost for in situ system and product assessment l Assuming not more than 10 data providers per year, $145, 000 20

USGS Implementation Plan – Quality Assurance and Assessment Team l Current Plan - Develop QA/QC guidelines l Future - Enhance and develop statistical QA/QC methods with recommended contract publishable statistics baseline and contract performance guidelines; GIO funded l Future Plan - Continue training in FY 08 and FY 09 – Continue with GIO funding – Need to establish a good web presence and documentation for the users l Approximate cost in FY 08 and FY 09 – $163, 000 21

USGS Implementation Plan – Contracting Guidelines and Acquisition Tool l Current Plan - Develop digital imagery evaluation tool, complete acquisition documentation, incorporate QA/QC guidelines; Establish the USGS IADIWG Acquisition Guideline for Digital Aerial Imagery u l LRS cost in FY 08 – Guidelines $88, 000 + IADIWG $79, 000 Future – Provide these components to GIO for operational sustainment 22

USGS Implementation Plan – Training l Training – Training of users, State Liaisons, conferences, workshops, … u GIO to continue training in FY 08 and FY 09 u Need to establish a good web presence and documentation for the users u Approximate cost in FY 08 and FY 09 – $141, 000 23

USGS NGPO requirements l Contracting Guidelines – Take over the contracting guidelines and support in FY 09 – input from DP and QA teams in FY 08 l Manufacturer Type Certification – GIO DP and QA team to follow results and status – GIO member part of the team - LRS funded l Data Provider Certification – GIO responsible for the process evaluation – DP team responsible - $152, 679 l Image Quality Assurance and Assessment Guidelines – GIO to provide QA team and support – deliverables due end of FY 08 - $ $163, 417 l l User Training - GIO to provide funding for training of partners - $141, 286 ($75, 000) Project management - GIO will provide associated project and team management support and project infrastructure costs (IT, facilities, management overhead - $123, 180 l Total Cost: $540, 900 24

USGS LRS requirements l Contracting Guidelines – Fund web page, tools, and enhance guidelines GIO responsible in FY 09 - $114, 250 l Manufacturers Certification - LRS funding required to support 4 manufacturer certifications and infrastructure - $142, 857 SIR + $57, 143 REI l Data Providers Certification – LRS to fund in situ range support, evaluation tools, and reporting - $147, 458 SIR ($75286 overguide)+ $35, 174 REI l Image Quality Assurance and Assessment Guidelines – GIO requirement l User Training - GIO to provide funding for training of partners l Project management - LRS will provide associated project and team management support and project infrastructure costs (IT, facilities, management overhead - $94036 l Total Cost: $470, 179 (includes MCGSC $90 K and a $75, 286 overguide) + Reimbursable income 25

Sioux Falls Range Established l l Three target extents High altitude: <1 m pixels Medium Altitude: < 1 ft pixels Low altitude: > 4 cm pixels Will be useful for high-res satellite characterization also Done with City of Sioux Falls, Army, and NGA Will be used to test new sensor types and certification of sensors types and data providers 26

EROS and Sioux Falls Ranges Upgraded l l New targets added US Army Corp of Engineers worked with EROS personnel to establish LIDAR targets on EROS range u l Surveyed to ± 2 cm in X, Y, Z 32 targets across EROS property l Similar ranges to be established around the US 27

Future Requirements for Digital Imagery l l l Interoperability and format guidelines Lidar tools Multi-sensor fusion tools In-situ geometric calibration Radiometric calibration processes 28

Future Plans l l Implement USGS QA Plan (FY 07 -FY 08) in US Prepare for NLIP and IFTN (FY 08) u l Work with International Agencies to establish common and collaborative processes u l l l USGS GPCS and USDA NAIP contracts Euro. SDR, Australia, Canada BC Ministry of Agriculture and Lands - small & medium format specifications Establish in-situ calibration ranges across US (FY 07) Complete in-situ calibration s/w methods (FY 07 -08) Continue new sensor/system assessment Additional radiometric research and lab (FY 08) Establish Program commitments for FY 08 and FY 09 29

Summary and Required Actions l l Work with George/Jim/Kari to establish the BASIS+ funding profile for Orthoimagery project in FY 08 Update RST project tasks in BASIS+ Provide USGS QA implementation plan for LRS and NGPO signature and publication Continue to utilize project teams across LRS and NGPO 30