GOES-R Data Distribution Thomas Renkevens (NOAA/NESDIS/OSD) 4 th GOES-R Users’ Conference Broomfield, CO Tuesday, May 2, 2006 1
Outline • Requirements • Instrument Summary and Data Evolution • PDRR Contract – Trade Studies • • • Users and CLASS Data Distribution Issues Examples of Current Distribution Systems GVAR vs GRB GOES-West Location Summary 2
GOES-R Observational Requirements Aerosol Detection Aerosol Particle Size Suspended Matter Volcanic Ash Aircraft Icing Threat Cloud Imagery Cloud & Moisture Imagery Cloud Base Height Cloud Layers / Heights & Thickness Cloud Ice Water Path Cloud Liquid Water Cloud Optical Depth Cloud Particle Size Distribution Cloud Top Phase Cloud Top Height Cloud Top Pressure Cloud Top Temperature Cloud Type Convection Initiation Enhanced "V"/Overshooting Top Detection Hurricane Intensity Imagery: All-Weather / Day - Night Surface Albedo Surface Emissivity Vegetation Fraction Vegetation Index Atmospheric Vertical Moisture Profile Atmospheric Vertical Temperature Profile Capping Inversion Information Currents Ocean Color Ocean Optical Properties Ocean Turbidity Sea & Lake Ice / Displacement & Direction Sea & Lake Ice / Age Sea & Lake Ice / Concentration Sea & Lake Ice / Extent & Characterization Derived Stability Indices Moisture Flux Pressure Profile Total Precipitable Water Total Water Content Clear Sky Masks Radiances Absorbed Shortwave Radiation Downward Longwave Radiation Downward Solar Insolation Reflected Solar Insolation Upward Longwave Radiation CO Concentration Ozone Total SO 2 Detection Derived Motion Winds Lightning Detection Low Cloud & Fog Turbulence Visibility Geomagnetic Field ABI – Advanced Baseline Imager Dust/Aerosol Probability of Rainfall Potential Rainfall Rate Microburst Wind Speed Potential Fire / Hot Spot Imagery Flood / Standing Water Land Surface (Skin) Temperature HES – Hyperspectral Environmental Suite SEISS – Space Env. In-Situ Suite SIS – Solar Instrument Suite Sea & Lake Ice / Extent & Edge Sea & Lake Ice / Surface Temp Sea & Lake Ice / Motion Sea & Lake Ice / Thickness Ice Cover / Landlocked Snow Cover Snow Depth Sea Surface Temps Energetic Heavy Ions Mag Electrons & Protons: Low Energy Mag Electrons & Protons: Med & High Energy Solar & Galactic Protons Solar Flux: EUV Solar Flux: X-Ray Solar Imagery: X-Ray GLM – GOES Lightning Mapper 3 Magnetometer
GOES-R Baseline Instruments to Meet User Requirements • Advanced Baseline Imager (ABI) – Monitors and tracks severe weather – Images clouds to support forecasts • Hyperspectral Environmental Suite (HES) – Provides atmospheric moisture and temperature profiles to support forecasts and climate monitoring – Monitors coastal regions for ecosystem health, water quality, coastal erosion, harmful algal blooms • Solar Imaging Suite (SIS) and Space Environmental In-Situ Suite (SEISS) – Images the sun and measures solar output to monitor solar storms (SIS) – Measures magnetic fields and charged particles (SEISS) – Enables early warnings for satellite and power grid operations, telecom services, astronauts, and airlines • Geostationary Lightning Mapper (GLM) – Detects lightning strikes as an indicator of severe storms 4
GOES Evolution I-P Combined I-M N-P R-Series 5
GOES-R PDRR Contracts and Trades KDP-B Internal government plus 12 industry-led System Architecture Studies KDP-C/D 3 System Program Definition & Risk Reduction Contracts 2005 Single System Prime Acquisition & Operations Contract 2007 Long lead instrument development initiated early Launch Readiness 2012 • Three Program Definition and Risk Reduction (PDRR) Phase contracts awarded October 22, 2005 • As part of PDRR Contracts, following Trades are under study: – Data Distribution (GFUL and GRB) • Distribution methods and content – Infrastructure Architecture and Interface • Interface with NESDIS facilities 6
Data Users MRD ID: 6487 “Discussion: There also two classes of data usage - operational and retrospective. Most users performing operational data usage requiring real-time data delivery forecast and warning services will receive their GOES-R data directly from the PD Grouping or from the GRB. There are other users performing near-real-time operational processing who could query the GOES-R database to “pull” data in different temporal, spatial, or spectral forms depending on a particular immediate needs…” MRD ID: 6491 “The GOES-R system shall make data available to the users portals. Discussion: Making data available to the users portals may occur by a number of means including the rebroadcast of the partial data set of the GRB as described in section 2. 10. 8. 3. 1 and options for GFUL (described in section 3. 5. 7. 4 on GOES-R Series Level 1 b Data) ranging from push and pull capability to GFUL distribution. Push implies sending data to the user on a subscription basis, typically delivered to the user on a regular basis. Pull means users request data, not necessarily on a regular basis. ” 7
Key Data Users and CLASS MRD ID: 5233 “Users shall include, at a minimum, NOAA's National Weather Service - including: NCEP Units of TPC in Miami, SPC in Norman, AWC in Kansas City, OPC, HPC, CPC, EMC in Camp Springs, SEC in Boulder, NCEP Modeling Centers in Fairmont West Virginia, NWSTG in Silver Spring and its backup at TBS, Do. D in AFWA in Omaha, FNMOC in Monterey, NESDIS in Camp Springs and Suitland; other portions of NOAA; Academia” MRD ID: 5171 “CLASS will distribute data and products to a wide variety of users ranging from scientists doing weather and environmental research to school children doing homework to other interested parties wanting a satellite image of a recent storm in their area. Users will access the CLASS site via the Internet using a standard web browser. The system will enable Users to search for the data of interest based on source, instrument, time, and location and will provide browse images as a search aid. Future enhancement to CLASS will potentially enable more advanced search capabilities such as natural language processing, browse animations, and category searches, e. g. , “tornados” or “clear days”. ” 8
Data Distribution Issues • From CONOPS (March 22, 2005) Section 3 – “The difference in processing and communications requirements between GOES-R and earlier GOES series necessitates a substantial increase in bandwidth, processing capability, and dissemination approaches. ” – “The large increase of GOES-R data may prevent the entire set of Level 1 b data, called GFUL, from being rebroadcast. A subset of these data, GRB, will be transmitted to the GOES-R satellite for rebroadcast to all users. Key users will receive GFUL data by TBD method. Table 3 -1 Comparison of Processing and Storage of GOES-R Series to GOES I –P (TBR) 9
Data Distribution Issues (continued) • Data distribution considerations under study: – Distribution methods for GFUL – Development of a product generation (PG) system that will meet production processing timelines and product latencies. • Contents of GRB under study: – Level 1 b and/or Products • Support of various data formats (MRD ID#6494) – “The PD shall process product formats including, at a minimum (TBR), GIF, Text, BUFR, GRIB, Binary, JPEG, Net. CDF, and Mc. IDAS files or their replacement file formats. ” • Other potential formats are under discussion • Some current distribution methods include – GVAR Direct Readout, NOAAPORT, Mc. IDAS, Unidata IDD 10
Current GVAR Direct Readout System Satellite “Downlink” (2. 6 Mbps) “GVAR” (2. 11 Mbps) Any GVAR site Users Wallops CDAS The GVAR is a combination of (mostly) imager and sounder (scaled) 11 radiances (no products). No data compression is used.
Known GVAR Sites More information on the GVAR system: http: //www. osd. noaa. gov/gvardownload. htm 12
NOAAPORT Distribution • Satellite Broadcast Network (SBN) – Provides broadcast and reliable multicast data transmission to field sites. • Transmitted data includes: Centrally collected radar data, GOES imagery, NCEP model data, field observations, and watches and warnings – DVB-S • Single channel solution. • Linearly scalable up to 43 Mbps, on demand 13
NOAAPORT Evolution • Current Architecture challenged to meet increasing data volumes, • AWIPS moving to Service Oriented Architecture (SOA) – Raytheon to implement J 2 EE Enterprise Service Bus • SOA will provide a more flexible and robust infrastructure for AWIPS • Data delivery and information architecture – Introduce a more flexible data retrieval paradigm • Move to “push-pull” data delivery paradigm – Expanding AWIPS beyond push capability (SBN) only – Exploring use of Open. Dap as a technology to enable a push-pull paradigm From Tuell et. al, 2006 AWIPS Then and Now The Evolution of AWIPS 22 nd International Conference on Interactive Information Processing Systems for 14 Meteorology, Oceanography, and Hydrology, Atlanta, Amer. Meteor. Soc.
Mc. IDAS Evolution Mc. IDAS Background • Mc. IDAS is a meteorologically oriented analysis and display package, originally developed by the University of Wisconsin's Space Science and Engineering Center, that emphasizes comprehensive capabilities for image processing of data from satellite-borne sensor systems, and that supports ADDE client/server access to remote data • A collection of user programs and libraries for visualizing and analyzing data • Support for all international environmental satellites, numerous other observational data and NWP output Reasons for Change • New, advanced instruments require more powerful functionality • NPOESS (VIIRS, Cr. IS); GOES-R (ABI, HES) – Imagers: fast, multiple channel combinations – Sounder: convolution of 2000+ hyperspectral channels – Both: co-registration with full radiance information and metadata Candidate Transition System • IDV: Integrated Data Viewer – Based on SSEC’s Vis. AD package – Has a data model that is inherently capable of working with standard data types both now and in the future – Developed for education community by Unidata – Designed to replace Mc. IDAS at Universities 15
What is the IDV? • Unidata’s newest scientific analysis and visualization tool • Freely available Java™ framework and reference application • Provides 2 - and 3 -D displays of geo-scientific data (plus, of course, animations) • Stand-alone or networked application • Built on Vis. AD library The Integrated Data Viewer (IDV) is a meteorologically oriented, platform-independent application for visualization and analysis. Developed using Java, Vis. AD and other component libraries, the IDV emphasizes interactive 3 D visualization and integration of 16 diverse data types.
What is Vis. AD? • • Open-source, Java library for building interactive and collaborative visualization and analysis tools Features include: – Powerful mathematical data model that embraces virtually any numerical data set – General display model that supports 2 - and 3 -D displays, multiple data views, direct manipulation – Adapters for multiple data formats (net. CDF, HDF-5, FITS, HDF-EOS, Mc. IDAS, Vis 5 D…) and access to remote data servers through HTTP, FTP, DODS/OPe. NDAP, and Open ADDE protocols – Support for data sharing and real-time collaboration among geographically distributed users ADDE Servers (Abstract Data Distribution Environment) • • • In use for nearly 10 years Sectorizing of geographic coverage and bands Compressed transmission TCP/IP protocol through ports 112 Variety of clients can access the servers Mc. IDAS Information From Dave Santek, Tom Whittaker, and Thomas Achtor. Space Science 17 and Engineering Center, University of Wisconsin in Email correspondence April 7, 2006
Mc. IDAS-X vs Mc. IDAS –V Example Mc. IDAS-X display of an operational derived product: Cloud Top Pressure. Image is directly imported into Mc. IDAS-V, along with navigation, graphics, and color table. 18
Unidata IDD (Internet Data Distribution) • • • The Unidata community of over 150 universities is a system for disseminating near real-time earth observations via the Internet… Unidata IDD is designed so a university can request that certain data sets be delivered to computers at their site as soon as they are available from the observing system. . . Satellite, radar, and derived product imagery are available to Unidata core sites in different datastreams delivered in the Unidata IDD: GOES-East and -West sectors for selected wavelength bands, created at the SSEC/University of Wisconsin-Madison, and GOES derived products created at the CIMSS/SSEC are available in the IDD UNIWISC (Unidata-Wisconsin) datastream 19
Unidata IDD Evolution • • • The IDD has grown to become the leading Internet 2 advanced-application and one of the top bandwidth users (http: //netflow. internet 2. edu/weekly/ ), currently delivering about 20 terabytes (TB) of data per week in the aggregate to participating institutions. The Unidata IDD has expanded from a US-centric delivery system to one that includes 13 countries on 5 continents. Most recently, the implementation of a four-node Linux cluster as a top-level IDD relay demonstrated the ability to relay significant amounts of data to downstream sites (Yoksas, et al, 2005). Live stress testing (testing conducted on an “operational” system already feeding data to 220 downstream connections) showed that the cluster was able to relay – on average – over 500 Mbps (5. 4 TB per day) of data to downstream sites during a three day trial without introduction of product latency. The limiting factor in this stress test was not the LDM software or cluster node performance, but, rather, not having more downstream connections. Peak relay data rates exceeding 900 Mbps showed that the limiting factor in the ability to relay data was the underlying gigabit network in UCAR. The successes of the LDM-6 have not deterred investigation of alternate approaches to data distribution by the Unidata Program Center. – Network News Transfer Protocol (NNTP) , Bit. Torrent, design of a new data relay system – This ongoing activity will provide the underpinnings of a next-generation IDD that will even better serve the international Unidata community. From Yoksas et. al, 2006 The Unidata Internet Data Distribution (IDD) System: A Decade of Development 22 nd International Conference on Interactive Information Processing Systems for Meteorology, Oceanography, and Hydrology, Atlanta, Amer. Meteor. Soc 20 http: //www. unidata. ucar. edu/software/idd/
Product Distribution Grouping • The Mission Requirements Document (MRD) defines the Product Distribution grouping as follows: MRD ID: 321 “The Product Distribution (PD) grouping includes distribution of level 1 b (GOES full data set), GOES-Rebroadcast data (GRB), and derived products to user portals while addressing interfaces with the user for accessing GOES data. ” Note: GOES-R distribution requirements state all level 1 b and derived product distribution will be through user portals 21
GOES Rebroadcast (GRB) • GRB system is extension of current GVAR system for GOES-R era • GRB system described here is a payload service, separate from any direct service (LRIT, DCPI/R, EMWIN, and SAR) – GRB is needed to make a large amount of data available to a wide range of users (geographically and in terms of data use) in a cost efficient manner • Main differences between GVAR and GRB: – GRB will be a larger data rate than the GVAR • 17 -24 Mbps vs 2. 11 Mbps – Due to new, large data rates for the instruments currently planned for GOESR, the GRB system will not realistically be able to transmit all level 1 b data without data compression • With input from PDRR vendors and key users, GRB content must be selected • Assumptions – User needs • All users and applications, both current and future, not known • If the data are available, user will work to gain access to it – Users will expect a similar (or higher) level of service – Communications capabilities will continue to evolve • Improving capabilities, technologies, and lower cost 22
GOES Rebroadcast (GRB) and GVAR MRD ID: 4430 “The GOES-Re-Broadcast (GRB) transponder shall support the rebroadcast of the ground processed weather data from the CDAS to a wide community of NWS and governmental and academic research organizations. ” MRD ID: 4431 “Data Content - Ground processed instrument data similar in content to the current GVAR data (the data content is transparent to the GRB transponder)” MRD ID: 6855 “The Ground Segment shall provide a GVAR rebroadcast format of a selected GOES-R series data subset through the GOES-N series satellites. Discussion: Current GVAR users that may not be able to upgrade to the GRB data receivers have expressed an interest in receipt of a GVAR-like data rebroadcast of GOES-R data in the 23 GOES-R timeframe using only GOES-N satellite series. ”
GOES-West view from 135º 24
GOES-West view from 138º 25
Summary • The great amount of information from the GOES-R series will offer both a continuation of current products and services, and provide improved or new capabilities. • Major improvements in GOES-R means major task in preparing for the change • Mission Requirements Document defines product distribution, users portals, key users, GRB • Implementation of data distribution and GRB content under study through PDRR phase • GOES-R instrument data rates increase by approximately two orders of magnitude over current instruments – Some exact instrument designs and data rates still under study • Need to continue to work with users and PDRR contractors to shape requirements, define GRB content 26
Alternate Distribution Methods? 27
Links to Additional Information • NOAA GOES-R Page – Links to CONOPS, GPRD, MRD – https: //osd. goes. noaa. gov/ • NOAA/NESDIS OSD Page – http: //www. osd. noaa. gov/ • NASA Industry Day – Links to Instrument Documentation – http: //goespoes. gsfc. nasa. gov/goesr_industry. htm • ABI Research at CIMSS – http: //cimss. ssec. wisc. edu/goes/abi/ • HES Research at CIMSS – http: //cimss. ssec. wisc. edu/goes/hes/ • ABI Documentation from NASA: – http: //goespoes. gsfc. nasa. gov/abihome. htm 28
Скачать презентацию GOES-R Data Distribution Thomas Renkevens NOAA NESDIS OSD 4 th
82e8cb2497723eda072e8a80ccfa291b.ppt