Status of HDF-EOS, Related Software and Tools Abe Taaheri, Raytheon IIS HDF & HDF-EOS Workshop XIV Champaign, IL Sep. 29, 2010 1
TOOLKIT / HDF-EOS Support • • Raytheon was awarded EOSDIS Evolution & Development (EED) contract in May 2010 Maintenance support for TOOLKIT/HDF-EOS is funded on EED through March 2015 • Current support § § § HDF-EOS 2 & 5 SDP & MTD Toolkits HDF-EOS 2 to Geo. TIFF Converter (HEG) HDF-EOS plug-in for HDFView Other Toolkit/HDF-EOS related software Toolkit/HDF-EOS/HEG helpdesk 2
ECS Support of HDF-EOS • Both HDF 4 and HDF 5 based flavors of HDF-EOS are part of ECS baseline. • OS’s: Solaris (10) Linux (32 & 64 -bit, including IA 64) Mac OS X (Power PC and Mac Intel) Cygwin Windows XP (MTD Toolkit, HDF-EOS 2 & 5, HEG, HDFView) • Compilers: Fortran 77/90 , g 77/pgf 90 C, C++, gcc, g++ Intel Fortran (SGI Altix), gfortran (Mac Intel) 3
Downloadable Software • DAACS are supported with – TOOLKIT/HDF-EOS & HEG deliveries during yearly DAAC releases or patch deliveries – On demand • Other users can download our software from – our tools’ web sites – directly from our ftp site: ftp: //edhs 1. gsfc. nasa. gov/edhs 4
Downloadable Software • SDP Toolkit (Science Data Processing Tools) • MTD Toolkit (Metadata, Time/ Date Tools) • HDF-EOS 2 (HDF 4 based) • HDF-EOS 5 (HDF 5 based) • heconvert (converts HDF 4 - based Grid/Point/Swath structures to HDF-EOS 5 equivalents) • Dump. Metadata (Dump HDF-EOS metadata) 5
Downloadable Applications • HEG (HDF-EOS 2 to Geo. TIFF conversion) • HDFView Plug-in, Java-based browser (HDF 4, HDF 5, HDF-EOS 2 and 5 access) • HE 5 View* (HDF 5 - based browser) • EOSView* (HDF 4 - based browser) * Dropped support for OS updates on July 2006 6
Software/Tools Brief description of TOOLKIT, HDF-EOS, And HDFView Plug-in 7
Toolkit • Science Data Production (SDP) Toolkit : collection of tools used by - Science software developers who produce code to process instrument data. - HDF-EOS users who need to • • Access metadata in HDF-EOS files Perform Time/Date conversion Access Digital Elevation Model Data Access Ancillary Data, such as Digital Chart of the World database (DCW), Olson World Ecosystem files, DEM data files, etc. 8
Toolkit (cont. ) • Tools provided in SDP Toolkit: - AA - CBP - CSC - CUC - DEM - EPH - GCT - IO - MEM - MET - PC - SMF - TD (Ancillary Data Access) (CELESTIAL Body Position) (Coordinate System Conversion) (Constant and Unit Conversions) (Digital Elevation Model Access) (Ephemeris Data Access) (Geo Coordinate Transformation) (Input Output (File I/O)) (Memory Management) (Metadata Access) (Process Control) (Status Message File (Error/Status)) (Time and Date Conversion) 9
Toolkit (cont. ) • MTD Toolkit: – a short version of SDP Toolkit – Tools provided: • MET (Metadata Access) • TD (Time and Date Conversion) – Also supported in windows 10
HDF-EOS • HDF-EOS: – an extension to HDF which standardizes storage of and access to common Earth Sciences data structures • By providing a single interface to data structures common to earth science • By providing a container for EOS inventory, archive and product specific metadata • By establishing a consistent relationship between geolocation and science data. 11
HDF-EOS 2 • HDF 4 - based, storage format for EOS standard products. • Used operationally by: – MODIS, MISR, ASTER, Landsat, AIRS – other EOS instruments • Support for Grid/Point/Swath structures 12
HDF-EOS 5 • HDF 5 - based, a complete rewrite of HDF-EOS 2 with a different user interface. • Used operationally by EOS Aura instrument teams • Designed to ‘resemble’ HDF-EOS 2 to the maximum extent possible. • Supports same data structures (and more: Zonal average) • Added prefix ‘HE 5_’ to HDF-EOS 2 functions to distinguish two libraries 13
HDFView Plug-in • HDFView: A Java-based browser from The HDF Group for HDF 4 and HDF 5 based data. • HDF-EOS plug-in – Extends HDFView to browse HDF-EOS data in both versions. – Is a plug-in module and does not change The HDFView functionality – Have identical look and feel when displaying HDF and HDF -EOS objects – Users can access a complete breakdown of HDF-EOS objects. e. g. grid info, projection info, dims, datafields, and attributes 14
HEG • HEG • • • Geo. TIFF conversion Multi-band Geo. TIFFs Swath to Grid conversion Spatial subsetting, Field Subsetting Reprojection Stitching, sub-sampling and more …. 15
Software Releases 16
Software Releases SDP Toolkit/HDF-EOS: • Last release – August 2010 – TOOLKIT 5. 2. 17 – HDF-EOS 2. 17, HDF- EOS 5. 1. 13 • with HDF 4. 2. 5, HDF 5 -1. 8. 5, Szip 2. 1, zlib 1. 2. 5 • Next release – (late 2010 – early 2011) – TOOLKIT 5. 2. 18 (with new 500 m DEM data) – HDF-EOS 2. 18, HDF-EOS 5. 1. 14 • with new releases of HDF 4 (2. 6), 5 (1. 8. 6) 17
Software Releases MTD Toolkit: • Last release – Dec 2006 – Version 5. 2. 13 • in-sync with SDP TOOLKIT 5. 2. 13 • with HDF 4. 2 r 0, HDF 5 -1. 6. 3, Szip 1. 2 • Next release – Oct 2010 – Version 5. 2. 17 • in-sync with SDP TOOLKIT 5. 2. 17 • with new releases of HDF 4 (2. 5), 5 (1. 8. 5) 18
Software Releases HEG (HDF-EOS 2 to Geo. TIFF Converter): • Last release – Dec 2009 – Version 2. 10 – Patch release Jan 2010 • Next release – Dec 2010 – Version 2. 11 19
Software Releases HDF-EOS Plug-in for HDFView: • Last release – Apr 2010 – For HDFView Version 2. 6 • Next release – Early 2011 – For HDFView Version 2. 7 to be released – Version 2. 6 plug-in works with HDFView 2. 7 20
Software Releases HECONVERT (HDF-EOS 2 to HDF-EOS 5 • • Converter): Last releases – 2001, 2004 No Versioning , Oct. 2008 Version 2. 0 Next release – Nov. 2010 Version 3. 0 » Fix for some missing attributes in the converted file Dump. Metadata (ASCII Dump of HDF-EOS metadata) • Last release – Dec. 2008 Version 1. 0 21
Recent Major Development/Fixes TOOLKIT: – XML metadata writing capability for inventory metadata in SDP toolkit (5. 2. 16 release). Added XML style sheet to new TOOLKIT distribution for metadata display with Internet Explorer and other browsers – Fix for writing metadata into an HDF file when multiple HDF files are open – libiconv delivery in tar files for Mac and Cygwin – Fix for setting correct endianness flag in linux for eph tools (Fix is for MOPITT eph files reading) 22
Recent Major Development/Fixes HDF-EOS 2: – Added Dimension scales for Swath and Grid Fix for: – Subsetting with GDdefboxregion results in an extra row and/or column in linux (round-off problem) – Inconsistency in dimensions between real dims and the values in the structure metadata – Problem on setting other compressions for swath and grid when disabling szip compression 23
Recent Major Development/Fixes – Segfault in creating swath when HDF files opened in read-only mode – Segfault in EHinquire function because of a VGroup with class "CDF 0. 0” – Segfault in GDapi (sincos redefinition problem) – Segfault in sample test grid subsetter – compinfo functions must return -1 only for errors (not for COMP_NONE) – GDrs 2 ll should fail for any projection other than BCEA 24
Recent Major Development/Fixes HDF-EOS 5: – Added Dimension scales to Swath, Grid, and Zonal Average fields – Fix for problem in setting fillvalue; Fillvalue is only for the field defined right after setting fillvalue 25
Recent Major Development/Fixes HDF-EOS Plug-in for HDFView: – Support for all features in HDFView 2. 6 – Correction for displayed lat/lon in a swath around the international date line – Bug fix for HDF 5 fillvalue and chunking info display – Bug fixes for saving HDF-EOS 5 files after modification 26
Recent Major Development/Fixes HEG: Added Product Support: – AMSR-E Daily 25 Km Sea Ice AE_SI 25 – AMSR-E Daily land AE_Land 3 – MOD 29 E 1 D: MODIS/Terra Sea Ice Extent & IST Daily L 3 Global 4 km EASE-Grid Day, Version 4 – Tested many new products and updated the list of Hegable MODIS, ASTER, AIRS, MISR, and AMSR-E products in HEG Users Guide and HEG web site 27
Recent Major Development/Fixes Enhancement: – Added a new projection, Cylindrical Equa Area (CEA), to support AMSR-E Daily Land product. – Pixel resampling support for gridded data with No. Reprojection – Outputting all fields in a file to a single HDF-EOS output when HDF-EOS output is requested. – Setting of projection parameters for multiband geotiff only once in the GUI 28
Recent Major Development/Fixes Bug fixes: – Pixel size unit correction for consistency between conversion and stitch GUIs. – Problem resolution with several products MOD 43 C 3, MOD 10 CM, MOD 29 P 1 N, AE_Rn. Gd, AE_Rain – Correcting sphere radius written to outputs for MODIS SIN grids – Resolving problems • with band selection in stitch GUI • with occasional core dump (runs that take long time to complete) in Datapool HEGserver 29
Future Plans TOOLKIT/HDF-EOS: • XML Metadata reading in TOOLKIT • Support for new versions of HDF 4 - HDF 5 • Update of DEM Tools for land-sea mask and new 500 m DEM data • Fix for getting correct compression info for a field in HDF-EOS 5, when GDcompinfo/SWcompinfo fails to get the info from Structure metadata • Support (or improvements) for 64 -bit platforms • Fix for Wrong values for some indices returned from SWindexinfo() in both hdfeos 2 and hdfeos 5 30
Future Plans • • • Store info in structural metadata somewhere else Auto Configuration for TOOLKIT RPMs for HDF-EOS and Toolkit Validation tool for HDF-EOS 5 files Fix for user/DAAC reported bugs 31
Future Plans (cont. ) HDF-EOS Plug-in for HDFView: • More functionalities : – – Cut/paste objects, modify/delete attributes. Create new grid/swath Support Zonal Average object for HDF-EOS 5 Add World Coast Line map to HDFView Plug-in HEG: – Performance improvement for Lambert Azimuthal Projection conversion – Support HEG in MAC Intel – Resolve reported bugs or requests from DAACs/Users 32
Availability • Access to libraries, applications, Documentation: TOOLKIT: http: //newsroom. gsfc. nasa. gov/sdptoolkit/toolkit. html HDFView: http: //newsroom. gsfc. nasa. gov/sdptoolkit/HDFView/ HDFView_hdfeos_plugin. html HEG: http: //newsroom. gsfc. nasa. gov/sdptoolkit/HEGHome. html HDF-EOS: http: //www. hdfeos. org/ ( new look, sample codes + HDF-EOS Forum ) • Email – Abe_Taaheri@raytheon. com – Landover_PGSTLKIT@raytheon. com 33
Thank You! 34
Скачать презентацию Status of HDF-EOS Related Software and Tools Abe
00da85b43566a29aeb35db5b862749f4.ppt