Status of the GHRSST GDAC Global Data Assembly

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO. DAAC Ed Armstrong, Jorge Vazquez and Andrew Bingham http: //ghrsst. jpl. nasa. gov

Overview of GDAC Tasks. . data management and distribution center for all GHRSST products Ingest project data files L 2 P, L 4, HR-DDS, MMR and MDB files from RDACs Stage on public FTP site for 40 days Deliver data and metadata to LTSRF (NODC) for 30 day old data Maintain metadata in a relational database (MMR) Provide web-base interface for data discovery and metadata query Fill ancillary fields for all L 2 P products Support GHRSST User Services and Applications Maintain a match-up database of in situ/satellite obs RDAC: MODIS L 2 P production 28 Nov 2005 US GHRSST meeting, RSMAS

Recent Activities Public GHRSST data release in August 2005 Interacted with OBPG and RSMAS to design the MODIS L 2 P processing architecture New Linux-based web and database severs, and development workstation integrated into PO. DAAC GDAC web pages enhanced Access to MMR improved Input for GDS rev 1. 7 including metadata revision IDL and C readers for L 2 P Implemented FGDC metadata and transfer to LTSRF Ingested NWP and model data streams for L 2 P ancillary fields Developing code and methodology for realtime ancillary filling Implemented user support through a customer management tool (Right. Now) First draft of a user requirements document 28 Nov 2005 US GHRSST meeting, RSMAS

GDAC System Architecture RDACs Medspiration (Europe) NGSST (Japan) Blue. Link (Australia) NAVOCEANO (U. S. ) RSS (U. S. ) PO. DAAC (U. S. ) GDAC (JPL U. S. ) OCEANIDS (Data Management) MMR/MDB NRT Users Data Tools L 4 PC (MIST – U. S. ) 28 Nov 2005 Archive (NODC – U. S. ) Users US GHRSST meeting, RSMAS

GDAC Web Page 28 Nov 2005 US GHRSST meeting, RSMAS

Master Metadata Repository 28 Nov 2005 US GHRSST meeting, RSMAS

Data Access FTP: podaac. jpl. nasa. gov/pub/GHRSST Data public since late August 2005 Organised by: datatype / sensor / RDAC / year / DOY Search interface through the MMR Future delivery mechanisms POET (L 2 subsetting) Open. DAP Live Access Server Subscription Data casting 28 Nov 2005 US GHRSST meeting, RSMAS

Data Ingest RDAC Datastream Status EUR L 2 P TMI L 2 P SEVRI L 2 P NAR 16 & 17 & 18 L 2 P AVHRR LAC (16 & 17) L 2 P AVHRR GAC (16 & 17) L 2 P ATSR L 2 P AMSR-E L 4 UHFnd HR-DDS Good Good No data since 25 Mar REMSS L 2 P AMSR-E L 2 P TMI Good, sporadic in Oct NAVO L 2 P AVHRR LAC (18) L 2 P AVHRR GAC (18) Expected Dec 2005 ? RSMAS/OBPG/JPL L 2 P MODIS_A In preparation NOAA NESDIS L 2 P GOES Unknown AUS (Bluelink) L 2 P AVHRR HRPT Unknown 28 Nov 2005 US GHRSST meeting, RSMAS

Data Latencies Datastream EUR EUR EUR L 2 P TMI L 2 P SEVRI L 2 P NAR 16 L 2 P NAR 17 L 2 P AVHRR LAC 16 L 2 P AVHRR LAC 17 L 2 P AVHRR GAC 16 L 2 P AVHRR GAC 17 L 2 P AATSR L 2 P AMSR-E L 4 UHFnd 28 Nov 2005 Avg (hr) Std Dev (hr) Max (hr) 20. 7 7. 2 7. 4 7. 0 8. 6 8. 5 7. 5 8. 1 9. 4 22. 2 11. 8 40. 4 16. 8 15. 1 12. 0 13. 3 12. 2 11. 2 13. 8 14. 6 48. 1 13. 4 1376 392 169 98 108 97 103 388 373 1377 108 US GHRSST meeting, RSMAS

Oceanids ingest statistics Granules Volume (GB) 28 Nov 2005 US GHRSST meeting, RSMAS

Ancillary fields JPL GDAC will fill the L 2 P ancillary fields of wind, solar insolation, sea ice and aerosols Some RDACs cannot fill these fields in an operational environment GDAC has accepted this additional responsibility at last GHRSST ST meeting Ingest numerical weather prediction and model data, i. e. , NCEP, NAAPS, FNMOC sea ice and SST A backup data stream will be implemented for each ancillary field (e. g. , ECMWF model output) Ancillary filling methodology and management has been conceptualized. • Readers for input fields being built 28 Nov 2005 US GHRSST meeting, RSMAS

NOAA LTSRF Interface Status After 30 days data are placed on a rolling stage for NOAA to pull FGDC metadata created “on the fly” through access to the MMR Formatted for the NOAA accession record (daily collection of same product files) Currently NOAA has implemented an automated retrieval of L 2 P data and FGDC metadata FGDC records for all L 2 P Medspiration data created through Nov 2005 Other products to follow 28 Nov 2005 US GHRSST meeting, RSMAS

MMR Status MMR (Master Metadata Repository) Fully transactional My. SQL relational database Schema finalized and implemented for FR and DSD metadata in Jan 2005 • New database server implemented in Jun 2005 Over 80, 000 DSD and FR records ingested since Feb 2005 • No operational failures Web search interface improved, access is very fast. Automated translation to FGDC metadata through XML style sheet conversion • Other standards can also be supported 28 Nov 2005 US GHRSST meeting, RSMAS

User services and applications Updated US GDAC User Requirements document GHRSST requirements GDAC system requirements RDAC requirements User requirements • E. g. , OOSes 28 Nov 2005 US GHRSST meeting, RSMAS

MODIS L 2 P Joint effort between Univ. of Miami RSMAS, NASA Ocean Biology Processing Group (OBPG) and JPL This is a new RDAC activity Initially concentrate on Aqua SST RDAC Division of Labor RSMAS: Algorithm and error statistics OBPG: 1 km L 2 SST production with some ancillary fields JPL GDAC: L 2 P integration with error statistics and ancillary fields Current Status OBPG has produced a realtime L 2 SST stream for MODIS Aqua (and Terra) RSMAS has developed a MODIS error “hypercube” JPL has tested throughput and acquisition from OPBG server 28 Nov 2005 US GHRSST meeting, RSMAS

Future activities. . . Ancillary fields filling Accept delivery of Linux 8 node cluster Installed in Jan 2006 Implement code for realtime filling • Investigating parallelization of processing MODIS L 2 P Finalize transfer between three parties Implement error statistics and L 2 P Schedule for Aqua and Terra SST L 2 P Interface to new RDACs NAVO NOAA NESDIS OBPG Update ICD 28 Nov 2005 US GHRSST meeting, RSMAS

. . . Future activities FGDC metadata Operational transfers for all GHRSST products (L 2 P, L 4) to LTSRFs User Services GHRSST product support Applications development Data delivery L 2 P subsetting in POET Open. DAP Improved search and delivery via MMR New technology infusion GDAC will be a test bed for the Earth Science Data Casting Project • a recently funded NASA ROSES activity for XML-based data management and delivery Implement the Matchup Database (MDB) with similar functionality as MMR 28 Nov 2005 US GHRSST meeting, RSMAS