CATENA Distributed Generic Processing Chain for Optical Satellite Imagery Processing Peter Reinartz, Thomas Krauß Remote Sensing Technology Institute Photogrammetry and Image Analysis ESA Workshop on Models for Scientific Exploitation of EO data Frascati, 2012 -10 -11
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 2 Why processing chains for higher level optical data processing? • Needed for: • Processing of large areas and large data volumes e. g. Image 2006/2009/2012, each time about 3500 scenes IRS/SPOT for whole Europe-38 • Processing of time series e. g. CCI-Fire, Meris/ATSR/SPOT-VGT for 1995 -2009, about 130. 000 scenes • Requirements: • Fully automatic processing of • Mass data from • Many optical sensors/satellites • Modular and easy re-configurable for many projects Image 2006, ~3500 IRS/SPOT scenes
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 3 CATENA – chain for fully automatic processing of optical satellite data CATENA Automatic and operational processing chain for processing of mass data Using global databases and reference data Support of native satellite image formats from SPOT 4/5, IRS-P 6 Liss 3/AWi. FS, ALOS AVNIR/PRISM, Ikonos, Quickbird, Rapid. Eye, World. View, Geo. Eye, Cartosat, Pleiades, Meris, ATSR, VGT, Modis, … input output Reference image DEM Image Matching Original image Sensor Model Refinement Orthorectification extract ground control points from global Reference databases perform parameter estimation Use global DEM database Atmospheric correction remove atmospheric influence Ortho image Thematic processing
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 4 CATENA – Use cases CATENA • Catena is – A chain of processing modules – Uses interface- and data standards – Usable as DIMS- or stand-alone-version • Example use cases are – Orthorectification in Image 2006, Image 2009, Urban. Atlas, . . . – DEM-Generation as service for Cartosat/Euromap – Stereoprocessing, Time series, CCI-Fire, . . .
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 5 CATENA – Requirements CATENA • Systems and Libraries: – Linux (tested on Cent. OS, Ubuntu) – XDibias (DLR in-house development) – Python, scipy, numpy – GDAL • Modules: – Must not be interactive (automatic processing chain!) – Preferable: UNIX C/C++ source code, python code – Possible: Java, Fortran, any standard UNIX (script) programming language – No commercial programming environments which require any kind of licenses!
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 6 CATENA – Interfaces CATENA • Input-Data: – Original Level-1 -satellite data containing all metadata – Processed data including required metadata • Modules: – Image data and metadata in standardized XDibias format – Modules wrap existing processors with configuration files and any image format supported by GDAL • Output: – Any image format supported by GDAL – Standardized export. xml containing meta- and processing info – JPG-Quicklooks, KML files, any other intermediate files
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 7 CATENA – Summary of Principles CATENA • Standardized image- and metadata formats • Standardized process flow organization • Processor follows ESA „Generic IPF Interface Specifications“ • Distributed computing and storage • Standardized Development and Deployment process • Guidelines for module development, documentation and deployment • ISO 9000 certification in process: external audit today (2012 -10 -11)
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 8 CATENA – System overview CATENA Modules and order defined in chain Processing chains Chain. . . Delivered data Web-Interface Import Module 1 CCI-Fire Delivery Ortho Ingestion Atmospheric Corr. Original data DEM-Generation Select processing chain and set parameters Module 2 Module 3 Module 4 Module. . . Export Processing control system DIMS-PSM or stand-alone Cleaned up after delivery Workspace Some modules need additional data Reference DB 1 Reference DB 2 Reference DB 3 Standardized image and meta data Each job gets processed in own space
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 9 CATENA – Grid computing CATENA crontab Reference data crontab work Node 2 Node 1 data work crontab work scene database DB crontab work Node 5 Node 3 Node 4 . . . crontab work crontab Server work crontab work Node 6 Node n web server crontab • Simply add new node by creating working directory and inserting CATENA into crontab
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 10 CATENA – Distributed Mass Storage CATENA • Distributed mass storage with access from each processing node is needed for automatic processing of time series or bulk data for: • Realized as easily expansible Scality storage ring: Data is stored automatically in three distributed copies in the ring, read-access also in parallel from three storage nodes.
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 11 Web-Interface of stand-alone version CATENA
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 12 Examples CATENA • Orthorectification geocoded, optionally atmospheric corrected satellite images for further thematic processing and emergency mapping • DEM generation generate DEMs and Ortho images from (multi) stereo satellite data
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 13 Processing chain: Orthorectification • Standard processing chain for most optical satellite data • Satellites acquire oblique images • Ephemeris and attitude not exactly known • Correct these using ground control points • from already existing geocoded images • Project satellite image on existing digital elevation model • from DEM database (e. g. SRTM) • Resample satellite image in requested projection and resolution CATENA
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 14 Processing chain: Orthorectification Workflow Original image CATENA Reference image DEM Matching Control points Improvement of orbit and attitude data Manually measured ground control points Generation of ortho image Delivery Quality check Ortho image Atmospheric Correction Thematic processing
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 15 Overall Geometric Accuracy Co Co ve rag e 1 Overall (~4000 scenes) mean accuracy w. r. t. reference data set: RMSEx/y ~ 10 m ( CE 64 ~14 m) ~0. 5 pixel size of resampled images ve rag e 2 Requirement: RMSE < 20 m Mean number of ICPs per scene for accuracy assessment: IRS-P 6: 5496 points / scene SPOT 4/5: 1360 points / scene Residual plots available RMSE X Y
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 16 Input Processing chain: DEM generation Workflow Images Metadata Matching DEM Generation Orthorectification Output Processing Orientation DEM Ortho • At least two images from same orbit • Good relative orientation required, <0. 5 px, Bundle block adjustment • Dense pixelwise Semi-Global Matching = Disparity map on original images • Reprojection of DEM to target coordiante system, Interpolation and filling of holes • Orthorectification of the original imagery
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 17 Processing chain: DEM generation Ortho image and DEM, London
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 18 Processing chain: DEM generation London DSM from 5 World. View-2 Images
Processing Chains for Optical Data • Thomas Krauß • 2012 -07 -27 • www. DLR. de • Slide 19 Processing chain: DEM generation K 2 World. View-2 Triple Stereo • Very steep terrain • Very detailed surface model 15° • Film: http: //www. dlr. de/dlr/desktopdefault. aspx/tabid-10212/332_read-921/ 0° -15°
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 20 Processing Chains for Optical Data Summary CATENA • Processing chain CATENA developed at the Remote Sensing Technology Institute of DLR for fully automatic processing of mass data from many different optical satellites • Already in use for many projects (Image 2006 -2012, Urban. Atlas, CCI-Fire, Cartosat-DEM-processor, Worldview-2 and Pleiades DEM generation, …) • Based on the general processing chain infrastructure CATENA including: • Modular system of processing Modules connected to Chains • Distributed parallel grid computing • Distributed mass storage • Easily expandable, e. g. : • A new processing chain for a new project • Adding normal Linux-PCs or virtual machines as new background processing nodes Contact: Thomas Krauß, DLR-IMF, Thomas. Krauss@dlr. de
Processing Chains for Optical Data • Thomas Krauß • 2012 -10 -11 • www. DLR. de • Slide 21 Thank you for your attention
Скачать презентацию CATENA Distributed Generic Processing Chain for Optical Satellite
1095a0ffe4be19fb378c9832207f8bd5.ppt