GODAE High Resolution Sea Surface Temperature Pilot Project

GODAE High Resolution Sea Surface Temperature Pilot Project GHRSST-PP and the NOAA-NASA Collaboration for Data Management and Reanalysis Kenneth Casey NOAA/NESDIS/NODC October 20, 2004 NOAA/NASA GHRSST Overview

GHRSST-PP Objectives • To deliver a new generation of operational SST • • • products To ensure that duplication of SST activities are minimized To implement an operationally efficient methodology for real time fusion of SST data To develop and foster considerable scientific and operational knowledge during the lifecycle of the GHRSST-PP NOAA-NASA GHRSST-PP Overview October 20, 2004 (2)

International Framework NOAA-NASA GHRSST-PP Overview October 20, 2004 (3)

International Framework RDACs RDAC and GDAC (near real time users) Long Term Stewardship and Reanalysis Facility (delayed mode users) NOAA-NASA GHRSST-PP Overview October 20, 2004 (4)

GHRSST-PP Science Team • Craig Donlon (Chair), Hadley Centre • • • for Climate Prediction and Research, Met Office UK Bill Emery, University of Colorado, USA Chelle Gentemann, Remote Sensing Systems, USA Chris Mutlow, Rutherford Appleton Laboratory, UK Doug May, Naval Oceanographic Office, USA Gary Wick, NOAA/OAR ETL, USA Ian Barton, CSIRO Marine Research, Australia Ian Robinson, Southampton Oceanography Centre, UK Jim Cummings, Naval Research Laboratory/US-GODAE, USA Richard Reynolds, NOAA/NESDIS NCDC, USA Neville Smith, BMRC, Australia • Hiroshi Kawamura NASDA/University of • • • Tohoku, Japan Nick Rayner, Hadley Centre for Climate Prediction and Research, Met Office UK Peter Minnett, RSMAS, University of Miami, USA Bob Evans, RSMAS, University of Miami, USA Pierre Le. Borgne, Meteo France O&SI SAF, France Andy Harris, NOAA/NESDIS ORA, USA Ed Armstrong, JPL PO. DAAC, USA Ken Casey, NOAA/NESDIS NODC, USA Jorge Vasquez, JPL, PO. DAAC, USA David Llewellyn-Jones, Univ. of Leicester, UK Andrew Bingham. JPL PO. DAAC, USA Jean-Francois Piolle, IFREMER, France Chris Merchant, Southampton Oceanography Centre, UK NOAA-NASA GHRSST-PP Overview October 20, 2004 (5)

GHRSST-PP Outputs • An operational data • • • stewardship, exchange and delivery system Standardized, real-time L 2 P SST products Real time L 4 SST (gap free, gridded) Reprocessed L 4 SST for Climate Data Records High Resolution Diagnostic Data Sets (HR-DDS) Matchup Data Base (MDB) Master Metadata Repository (MMR) NOAA-NASA GHRSST-PP Overview October 20, 2004 (6) L 4 L 2 P

Users and Applications NOAA-NASA GHRSST-PP Overview October 20, 2004 (7)

Users and Applications Sampling from one RDACs User Requirement’s Document: • • • • MERCATOR MET OFFICE, UK, FORECASTING OCEAN ASSIMILATION MODEL (FOAM) MEDITERRANEAN OCEAN FORECASTING SYSTEM (MFS) EUROPEAN CENTRE FOR MEDIUM RANGE WEATHER FORECASTING TOWARDS AN OPERATIONAL PREDICTION SYSTEM FOR THE NORTH ATLANTIC EUROPEAN COASTAL ZONES (TOPAZ) BALTIC OPERATIONAL OCEANOGRAPHIC SYSTEM (BOOS) NORWEGIAN METEOROLOGICAL INSTITUTE (MET. NO) DANISH METEOROLOGICAL INSTITUTE ROYAL DANISH ADMINISTRATION OF NAVIGATION AND HYDROGRAPHY NORWEGIAN POLAR INSTITUTE NORWEGIAN MET. INSTITUTE’S MARINE FORECASTING CENTRE BLACK SEA GOOS WMO AFFILIATED INTERNATIONAL PROJECTS PROUDMAN OCEANOGRAPHIC LABORATORY NOAA-NASA GHRSST-PP Overview October 20, 2004 (8)

GHRSST Overview Talk A Collaborative Effort Between NOAA/NODC and NASA/PO. DAAC Team Manager: Pat Liggett October 20 th, 2004 Deputy Manager: Robert Benada Sue Heinz: Raytheon Manager Jorge Vazquez Edward Armstrong Andrew Bingham Chris Finch 2004 October 20, Rosanna Sumagaysay NOAA/NASA GHRSST Overview

Outline • GHRSST Processing System – – – RDACs, GDAC, Archive Data Products Data Flow Oceanids Data Volumes • GHRSST Data Management – – – MMR MDB User Services • NASA/NOAA Significance NOAA-NASA GHRSST-PP Overview October 20, 2004 (10)

GHRSST Processing System RDACs, GDAC & Archive NOAA-NASA GHRSST-PP Overview October 20, 2004 (11)

GHRSST Processing System Data Products NOAA-NASA GHRSST-PP Overview October 20, 2004 (12)

GHRSST Processing System Data Products • Level 2 Preprocessed data (L 2 P) – Level 2 data from each sensor in a standard netcdf format that has complete error statistics associated with each sensor. • High Resolution Diagnostic Data Sets (HR-DDS) – L 2 P data extracted in given predefined boxes to be used for validation studies. All data from different sensors co-located on common grid. NOAA-NASA GHRSST-PP Overview October 20, 2004 (13)

GHRSST Processing System Data Products Using AIRS Data Within the GHRSST-PP • Recommended data products for Near Realtime SST fields – L 2 SST, Cloud Fraction, Internal SST Error Estimate (rms uncertainty) products for GHRSST-PP Diagnostic Data Studies – IR Spectral Radiance and Cloud. Cleared Radiance L 2, L 3 SST – Boundary layer moisture/temperature (total column and layered) – Surface emissivity – Cloud fraction and cloud-top pressure – Cloud flags (including aerosol) Source: Denise Hagan, JPL NOAA-NASA GHRSST-PP Overview October 20, 2004 (14)

GHRSST Processing System Data Products • Example of Single Sensor Error Characterization for AIRS based on comparison with shipboard in situ MAERI data (Caribbean): Mean Bias (N=120) Map of Global Sites for GHRSST Diagnostic Data Sets RMS (N=120) Source: Denise Hagan, JPL – AIRS Valuable Reference Sensor for GHRSST Diagnostic Data Sets and SST Error Assessment NOAA-NASA GHRSST-PP Overview October 20, 2004 (15)

GHRSST Processing System Data Products • Level 4 Merged Analyzed SST products – 10 km global product (Remote Sensing Systems) – 2 km very high resolution regional products, including areas such as the Mediterranean NOAA-NASA GHRSST-PP Overview October 20, 2004 (16)

GHRSST Processing System - OCEANIDS • Autonomous data stream ingest, management and distribution • Built-in redundancy • GUI for monitoring data flows • Provides instant anomaly messaging to DAAC operators • Manage over 50 data streams (50, 000 files/month – 1 TB/month) • OCEANIDS is enabling the DAAC to work with near real time data streams and serve customers such as COOS and IOOS. NOAA-NASA GHRSST-PP Overview October 20, 2004 (17)

GHRSST Processing System – OCEANIDS – Data Stream Interfaces RDACs Medspiration (Europe) NGSST (Japan) Blue. Link (Australia) NAVOCEANO (U. S. ) MISST (U. S. ) PO. DAAC (U. S. ) User Interface GDAC (JPL U. S. ) OCEANIDS (Data Management) MMR/MDB Near Real Time Users Data Tools L 4 (MISST – U. S. ) Delayed mode Users Archive (NODC – U. S. ) NOAA-NASA GHRSST-PP Overview October 20, 2004 (18)

GHRSST Processing System – Data Volumes NOAA-NASA GHRSST-PP Overview October 20, 2004 (19)

GHRSST Data Management Master Metadata Repository Two types of metadata records based on NASA DIF format: • Data Set Record (DSD): – Information common to all files (i. e. , same sensor, data center etc. ) – Static: Generally prepared once for each data set • File Record (FR): – File specific information (e. g. , parameter, time and space etc. ) – Dynamic: Prepared on a file -by-file basis by each Regional Data Assembly Center (RDAC) • Both types integrated into a relational database: the Master Metadata Repository (MMR). NOAA-NASA GHRSST-PP Overview October 20, 2004 (20)

GHRSST Data Management Master Metadata Repository ta me RDAC submission [Perl scripts] Create/update/modify my. SQL tables [SQL] Pass QA Parse meta [QA check] Fail QA – return to source Metadata queries by external user NOAA-NASA GHRSST-PP Overview October 20, 2004 (21) GDAC submission

GHRSST Data Management Matchup Database (MDB) • Record of in situ/satellite matchups from buoys and other in situ platforms – Source for satellite sensor error statistics • Not an explicit metadata record but captures • actual data (5 x 5 grids of matchup information) These databases will facilitate the data discovery of multiple satellite platforms and products for the customer NOAA-NASA GHRSST-PP Overview October 20, 2004 (22)

GHRSST Data Management Matchup Database (MDB) Matchups from Pathfinder SST • 1996 -1999 • 1985 -1988 NOAA-NASA GHRSST-PP Overview October 20, 2004 (23)

GHRSST Data Management User Services • GDAC Customer Relationship Management (CRM) Plan – – Goals Target Audiences Strategies & Activities Evaluation Mechanisms NOAA-NASA GHRSST-PP Overview October 20, 2004 (24)

GHRSST Data Management User Services • Goals – Leverage existing PO. DAAC CRM infrastructure to ensure a high level of customer satisfaction – Fully integrate RDACs, GDAC and NODC into the Customer Knowledge Base for 24/7 support – Implement and distribute metrics developed by the GHRSST Project Office into the GDAC reporting system – Develop and maintain web and information access portal unique to GHRSST-PP NOAA-NASA GHRSST-PP Overview October 20, 2004 (25)

GHRSST Data Management User Services • Target Audiences – Customer Base – – – GHRSST Project RDACs Management Stakeholders • JPL, PO. DAAC, NODC, NOAA, NASA, – – – L 2 P Data Operational Users L 4 Public Data Customers NODC Long Term Archive NOAA-NASA GHRSST-PP Overview October 20, 2004 (26)

GHRSST Data Management User Services • Strategies and Activities – GHRSST-PP Web Portal – data and information access – 24/7 Knowledge Base Customer Support – Customer access to all communication channels: Email, telephone, fax, web. – Measures of Effectiveness-provide statistics report to stakeholders – Privacy policies – Operational communications – Solicit customer feedback and implement process improvement NOAA-NASA GHRSST-PP Overview October 20, 2004 (27)

GHRSST Data Management User Services Evaluation Mechanisms – monthly and quarterly analytic reports to Stakeholders • Provide monthly analytic reports to stakeholders on customer statistics – – Number of Customers accessing data via electronic transfer, subsetting tools Characterization of customers Number of customer interactions Results of closed end surveys to customers accessing data • Provide monthly statistics on data access, availability and transfer: – Number of successful data transfers – Number of data files served (volume? ) – FTP statistics • Number of Users, • Number of Files downloaded • Data volume transferred • Provide data quality metrics – Quantify number of customers providing feedback for validation studies against climatologies, insitu data, etc. NOAA-NASA GHRSST-PP Overview October 20, 2004 (28)

GHRSST Data Management User Services GDAC CRM Overview NOAA NASA GHRSSTPP JPL ESA Regional Data Assembly Centers/ Value Added Providers/LTA ·Medspiration ·RSS ·NAVOCEANO ·Bluelink ·JMA ·MODIS ·MISST ·Naval Research Monterey ·NODC GDAC Customer Relationship Data User Communities Discovery Data Access Building KB Queries Applications NOAA-NASA GHRSST-PP Overview October 20, 2004 (29)

NOAA-NASA Significance Why is all of this important? • SST fundamental parameter for understanding climate • change. New generation of SSTs used in data assimilation must come with full error characterizations. Directly related to NASA’s Earth Science Enterprise Focus themes. Includes: § “How can predictions of climate variability and change be improved”? § “How can weather forecast duration and reliability be improved by new space-based observations, data assimilation, and modeling? ” § “The NASA Earth Science Applications Theme was created to accelerate and expand the delivery of Earth system science research results to serve decision support for American citizens and the world. ” NOAA-NASA GHRSST-PP Overview October 20, 2004 (30)

NOAA-NASA Significance Why is all of this important? • Will also support themes dealing with Carbon Cycle, • • • Coastal Management, Water and Energy Cycles Will support themes dealing with improved data management, including standardization of data formats, metadata, user support, and integration of new technologies for faster data transfer. Also addresses all four of NOAA’s Goals: Climate, Ecosystems, Weather and Water, Commerce and Transportation And…GHRSST is IOOS-compliant!! NOAA-NASA GHRSST-PP Overview October 20, 2004 (31)

New Technologies • Aspera: Enables large data set transfers over any network regardless of network conditions and distance. • Right. Now: Enables user and RDAC access to knowledge base system for maximizing customer communication and satisfaction NOAA-NASA GHRSST-PP Overview October 20, 2004 (32)

Conclusion… This is a great example of NOAA and NASA working together to better meet the needs of our shared community of users! NOAA-NASA GHRSST-PP Overview October 20, 2004 (33)