RAINEX Planning Meeting Miami FL RAINEX Data Management

RAINEX Planning Meeting, Miami, FL RAINEX Data Management UCAR Joint Office for Science Support José Meitín Jim Moore Dick Dirks

RAINEX Planning Meeting, Miami, FL JOSS Data Management • Early involvement in project planning • Involvement with PIs to develop data management strategy • Consistent implementation of strategy for lifetime of project and beyond • Reliable and efficient archive and distribution system • Easy and efficient access to datasets by broader community UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL Data Management Considerations • Develop a consistent Data Policy • Consider Data Types Formats and Documentation (metadata) • Data Collection strategies (operations support, supplementary data) • Real-time Requirements ( data calibration, intercomparison, and needs for operational decision making) • Data Quality Control and Processing responsibilities • Data Archive structure, functions and responsibilities • Data Distribution protocols (user interface, metadata inventories, browse products) • Collaboration with other programs and data centers (interoperability)

RAINEX Planning Meeting, Miami, FL

RAINEX Planning Meeting, Miami, FL JOSS Data Services • • Data Questionnaire Data Management Plan Project Web Pages Real-time Operations and Data Ingest Field Data Catalog Quality control and dataset development Interactive Data Archive and Distribution Special media products (e. g. CD-Roms) UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL JOSS Data Services Data Questionnaire UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL JOSS Data Services • Project Web Pages

RAINEX Planning Meeting, Miami, FL JOSS Data Services • Real-time Operations Support and Data Ingest Satellite Ground Station Forecast Products Numerical Model Analyses Operational Data Streams UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL GTS METAR Observation Locations 6 -10 November 2004

RAINEX Planning Meeting, Miami, FL GTS Ship and Buoy Observation Locations 6 -10 November 2004

RAINEX Planning Meeting, Miami, FL Coastal and Marine Observation Locations * * o o NDBC Buoy CMAN * NWLON PORTS o TCOON (TX) TABS (TX) + LUMCON (LA) o WAVCIS (LA) + COMPS (FL)

RAINEX Planning Meeting, Miami, FL Land Observation Locations * * * o + ASOS AWOS modem GAEMN (GA) GFC (GA) o + o LAIS (LA) TX ET (TX) FAWN (FL) TNRCC (TX) ALMESO (AL)

RAINEX Planning Meeting, Miami, FL JOSS Data Services • Field Data Catalog (WWW based) Daily Mission Reports Operations Summary Status Reports Data Analysis Products Authoring Tools UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL JOSS Data Services • Quality control and dataset development Processing of research-quality, high resolution data Processing of standard operational data streams Compatible formats Composite datasets and data integration UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL Composite Data Sets at UCAR/JOSS A composite dataset is a collection (over some time period and region) of similar data (e. g. surface meteorological) from a variety of sources, put into a common format, and passed through a uniform quality control. Why does UCAR/JOSS develop composites? - Provides data in a uniformat with QC. - Allows determination of network/site problems. - Useful for model applications. - Prevents duplication of effort.

RAINEX Planning Meeting, Miami, FL KVII CVT ARM CVT LTER CVT AWOS CVT HRLY Visual ASOS CVT HRLY Visual ABLE CVT HRLY Visual HPCN CVT Visual SCAN CVT Visual NMSU CVT Visual GWMD CVT Visual MADIS CVT Visual Ameriflux CVT Visual NCAR (4) CVT HRLY Visual OKMESO CVT HRLY Visual WTXMESO CVT HRLY Visual Others (7) CVT Visual HRLY Visual IHOP_2002 Hourly Surface Composite Development Gross Limit Checks MERGE Horizontal QC Examine Statistics Visual Hourly Surface Composite

RAINEX Planning Meeting, Miami, FL Aircraft Flight Level Datasets NCAR / RAF Net. CDF NOAA / AOC “standard” format JOSS RAINEX Archive Integrated RAINEX Net. CDF JOSS convert “type 5” to Net. CDF

RAINEX Planning Meeting, Miami, FL JOSS DATA MANAGEMENT TOOLS Field Catalog In-field tool to ingest and display operational and preliminary research data and project documentation for making real-time decisions and evaluating project progress Features: • Daily Mission Reports • Operations Summary Interactive Data Archive/Access System Primary means for all project scientists and researchers to browse and retrieve data from any JOSS-supported projects Features: • Long-term field project data archival and distribution • Facility Status Reports • Interactive data browsing, subsetting, and format conversion • Data Analysis Products • Remote Archive Access • Authoring Tools • Value-added datasets • Web-based access • Data documentation

RAINEX Planning Meeting, Miami, FL JOSS Data Services http: //www. joss. ucar. edu/codiac/ • Interactive Data Archive and Distribution Ease of Data Access (web based) Data available via FTP, magnetic tape, CD-Rom Metadata descriptions Data Browse, Subsetting, Format Translation Capabilities Quality Control Distributed Data Archives Customized data ordering and reformat UCAR Office Of Programs Joint Office for Science Support

RAINEX Planning Meeting, Miami, FL RAINEX Data Management Issues: • Data Policy • Data Management Plan • Real Time Data Needs (model, satellite) • Distributed Archive Responsibilities • Special Data Processing (surface, upper-air, airborne radar aircraft flight-level)

RAINEX Planning Meeting, Miami, FL http: //www. joss. ucar. edu/ meitin@ucar. edu jmoore@ucar. edu dirks 2@ucar. edu UCAR Office Of Programs Joint Office for Science Support

Acronyms PORTS – Physical Oceanographic Real-Time System NWLON – National Water Level Observation Network TCOON – Texas Coastal Ocean Observation Network (TAMU – Corpus) TABS – Texas Automated Buoy System (TAMU) LUMCON – Louisiana Universities Marine Consortium WAVCIS – Wave Current Surge Information System (LSU) COMPS – Coastal Ocean Monitoring and Prediction System (USF) GAEMN – Georgia Automated Environmental Monitoring Network (U of Georgia) GFC – Georgia Forestry Commission ALMESO – Alabama Mesonet (Auburn U) LAIS – Louisiana Agriclimatic Information System (LSU) TX ET – Texas Evapotranspiration Network (TAMU) FAWN – Florida Automated Weather Network (U of Florida) TNRCC – Texas Natural Resources Conservation Commission