AGU Poster IN 21 B-1049 AGU Fall Meeting

Зарегистрируйтесь, чтобы просмотреть полный документ!

AGU Poster IN 21 B-1049 AGU Fall Meeting December 15, 2009 The Data Management System for the Shipboard Automated Meteorological and Oceanographic System (SAMOS) Initiative http: //rvdata. us/, http: //samos. coaps. fsu. edu/ Shawn R. Smith 1, Robert Arko 3, Mark A. Bourassa 1, 2, Jiangyi Hu 1, Michael Mc. Donald 1, Jacob Rettig 1, and Jeremy Rolph 1 1 Center SAMOS Objective The Rolling Deck to Repository (R 2 R) project was recently launched with the ambitious goal of documenting “routine underway data” from the US academic research fleet and delivering those data to established national archives. Data distributions will be submitted by 18 operating institutions for 30 vessels, from hundreds of cruises per year. Partnership with SAMOS data assembly center In 2003, NOAA established a data assembly center (DAC) at the Florida State University to provide data stewardship for underway meteorological and near-surface oceanographic data collected by research vessels. A new partnership has been established between the R 2 R project and the DAC to extend SAMOS data stewardship to the US academic fleet. SAMOS Data Management System To collect, quality evaluate, distribute, and ensure future access (via national archives) to underway meteorological and near-surface ocean data collected on research vessels Overall, the data management system is a series of automated processes coupled to an SQL database. The database stores vessel metadata profiles, ship-specific processing parameters, file tracking and version control tags, and data quality flags. SAMOS Data The SAMOS data assembly center (DAC) receives voluntary contributions from 23 vessels (as of Dec 2009; Fig. 1). ASCII to Net. CDF Quality Control Preliminary Data Posted SASSI: Statistical Quality Control Analyst Notification On a 10 day delay, all files for a single ship and observation day are combined (allowing for late file receipt) Analyst Notification Problem Secondary automated QC locates spikes, steps, and highly variable observations and results in intermediate product. Visual Quality Control Save & Post Resulting research quality files are posted for users. Preliminary processing is fully automated. After verifying source and format of incoming message, data are converted to network common data form and merged with vessel specific metadata from database. Table 1 Automated QC checks data for valid ranges, ship speed, and location; agreement with a climatology; temporal sequence, and physical consistency. NODC Archive The final stage of the SAMOS DMS is to submit the data and metadata to the National Oceanographic Data Center. On a monthly basis, all original, preliminary, intermediate, and research-quality SAMOS data files are uploaded to NODC. Preliminary files are posted to web/ftp/THREDS servers. Analyst notifies operators at sea when problems are detected. 1. 2. 5. Trained data quality analyst uses a graphical user interface to add, modify, or remove QC flags. Internet RSYNC In 2009, the SAMOS DAC forged a partnership with the R 2 R to expand SAMOS data stewardship to all vessels in the US academic fleet. 3. 4. NOAA supports visual QC for select vessels Monthly Data SAMOS-R 2 R Partnership NODC develops FGDC metadata records and provides longterm access and stewardship. Objectives Initiate transfers of relevant parameters (Table 1) from all US academic fleet vessels Increase data transfer frequency to meet needs of operational weather forecasting community (at least every 6 hours) Conduct fully automated QC Provide operators with QC feedback and recommendations for sensor deployment Develop automated protocol for ship-to-shore metadata transfers Standards The SAMOS DAC will move to adopt R 2 R vocabularies for vessel names, ports, etc. The DAC continues to move towards the Climate and Forecast (CF) network common data form Plan to utilize R 2 R cruise inventory and cross populate vessel profiles Proposed Data Transfer Mechanism Two options: 1. Large operators with sufficient IT staff will continue to use current SAMOS data transfer protocol. 2. SAMOS-R 2 R to explore development of software/hardware to be installed on vessels • This protocol will support smaller operators • Recommended by UNOLS Research Vessel Technical Enhancement Committee • Input from technical community welcome. Metadata Challenge Since its inception, the SAMOS DAC has endeavored to collect extensive ship and instrument metadata, including digital imagery (Fig. 4), to meet the scientific goals our user community. Initially, metadata was requested from recruited vessels using simple forms that could be transmitted via email (Fig. 2). R 2 R Real-time Data Protocol The Rolling Deck to Repository Project acknowledges support from the National Science Foundation (NSF), Oceanographic Instrumentation and Technical Services (OITS) Program. Base support for the SAMOS data center is provided by NOAA’s Office of Climate Observation. OK Cron/Manual Trigger Merge Data File Merge handles temporal duplicates using preliminary QC Email receipt by the DAC triggers automated SAMOS processing (white list controlled). Check & Verify The one-minute interval of SAMOS data and the tendency of research vessels to operate outside of routine shipping lanes makes the observations ideal for satellite and model validation and calibration. Presently, SAMOS data are acquired directly from research vessels at sea via a daily email transfer protocol. The DMS automatically tracks progress of the daily data acquisition and quality control (QC), stores metadata on instrumentation and ships, and provides data monitoring capability via a user-friendly web interface. An SQL database stores essential parameters to support tracking, data QC, and version control throughout the process. The SAMOS DAC plans to develop new protocols to transfer underway meteorology and near-surface oceanographic data from the academic fleet to shore. Ideally, the protocol will allow the DAC to either transmit higher frequency (e. g. , 1 Hz) meteorological and surface oceanic samples from participating vessels to real-time servers at R 2 R or develop software/hardware that can be installed on the vessels. Under either protocol, the SAMOS DAC would support data reduction (averaging), shore-side monitoring, quality control, metadata acquisition, data distribution, and archival at a national data center. Fig. 1: SAMOS ship tracks for FY 2008 and FY 2009 color coded by operator. The Australians provide the only international data via the IMOS project. User Community The SAMOS Data Management System SAMOS Processing Transmissions nominally sent at 0000 UTC and include all observations for the previous day. Parameters collected (Table 1) vary from vessel to vessel, but must include observation time, position, and units. Other uses include: • Developing satellite retrieval algorithms • Air-sea interaction studies • Ocean process studies • Primary ocean production via radiative processes • Validation of operational marine forecasts Email Analyst Feedback Currently, recruited SAMOS vessels use an email protocol to transmit 1 -min. averaged data to the DAC The SAMOS Data Center The DAC for has developed and implemented an automated data management system (DMS) that collects, formats, quality controls, distributes, and archives near real -time surface marine data from research vessels. A SAMOS is a computerized data logging system that continuously records navigational (ship’s position, course, speed, and heading), meteorological (winds, air temperature, pressure, moisture, rainfall, and radiation), and near-surface oceanographic (sea temperature, salinity, conductivity, florescence) parameters while the vessel is at sea. The SAMOS initiative relies on the high-quality instrumentation purchased and deployed by the research vessel operators and does not provide instrumentation to the vessels. Currently, the SAMOS initiative receives measurements recorded at 1 -min intervals and derived from higher frequency samples (on the order of 1 Hz). In 2009, 23 research vessels are providing routine SAMOS observations to the DAC. Satellite Broadband (e. g. , Hi. Seas. Net) Ship Observations are from automated instrument systems operated and maintained by the vessel’s home institution SAMOS observations are one-minute average values derived from higher frequency (~1 Hz) instrument samples for Ocean-Atmospheric Prediction Studies, Florida State University 2 Department of Meteorology, Florida State University 3 Lamont-Doherty Earth Observatory of Columbia University Although operators may fill out forms initially, they still required tedious entry into our database and updates were difficult to obtain. New web-based forms (Fig. 3) were developed to allow operators to directly enter and update their metadata in our database. Fig. 2: SAMOS vessel and instrument metadata forms (Word version). The web forms are password protected and all updates require approval by the DAC. Web forms eased data entry, but are mostly used by DAC staff. In partnership with the R 2 R, the DAC will move to develop a protocol to automatically transfer necessary metadata along with the physical observations from ship to shore on a regular schedule It Takes a Team Researchers Students Technicians Data Managers Fig. 3: Web-based SAMOS ship and instrument metadata forms showing select data from the R/V Atlantis. Fig. 4: Examples of digital imagery from Select SAMOS vessels. (top) metadata collage from NOAA’s Okeanos Explorer; (lower left) instrument mast from the L. M. Gould; (right) Frozen anemometer on USCG Healy.

Скачать презентацию AGU Poster IN 21 B-1049 AGU Fall Meeting

b0f6585672da52fc76961e116f38f14a.ppt

Количество слайдов: 1