Web Mapping Weather Data Feeds Bob Wilson Bureau

Web Mapping Weather Data Feeds Bob Wilson Bureau of Meteorology Presentation for: AFAC-2010 Darwin http: //www. afac 2010. org. au http: //www. afac. com. au 2010/09/08

Outline § Background § Web Mapping Services / Architectures § What is the GIS 2 Web Project § What data layers are now available? § Accessing the Operational System § Future plans

The Bureau § § The Bureau was established in 1907 and Operates Largely under the Meteorology Act 1955 / 2008 and the Water Act. It operates within the Context of the WMO and the Australian Weather and Oceans Handbook. The Bureau areas of responsibilities include Forecasts, Weather (including Space weather), Climate, Oceans (including Tides and Tsunami) and Hydrology and National Water Accounting. It has over 1600 staff, Regional Offices in all states and over 60 Offices around Australian, Antarctica and Islands. IDDP 0004 Australian Government Bureau of Meteorology Northern Territory Darwin Regional Forecasting Centre CANCELLATION OF FIRE WEATHER WARNING For the Darwin and Rural Area Issued by the BUREAU OF METEOROLOGY, DARWIN at 1615 CST Tuesday 7 SEPTEMBER 2010 for the rest of today. Fire Danger of the Darwin and Rural Area has eased - no further warnings will be issued. DARWIN Regional Forecasting Centre.

Functions under the Meteorology Act 6 Functions of the Bureau (2) (1) The functions of the Bureau are: (a) the taking and recording of meteorological observations and other observations required for the purposes of meteorology; and (b) the forecasting of weather and of the state of the atmosphere; and (c) the issue of warnings of gales, storms and other weather conditions likely to endanger life or property, including weather conditions likely to give rise to floods or bush fires; and (d) the supply of meteorological information; and (e) the publication of meteorological reports and bulletins; and (f) the promotion of the use of meteorological information; and (g) the promotion of the advancement of meteorological science, by means of meteorological research and investigation or otherwise; and (h) the furnishing of advice on meteorological matters; and (i) co‑operation with the authority administering the meteorological service of any other country (including a Territory specified under subsection 4(2)) in relation to any of the matters specified in the preceding paragraphs of this subsection; and (j) such other functions as are conferred on the Bureau by any other Act. The Bureau shall perform its functions under this Act in the public interest generally and in particular: (a) for the purposes of the Defence Force; (b) for the purposes of navigation and shipping and of civil aviation; and (c) for the purpose of assisting persons and authorities engaged in primary production, industry, trade and commerce.

Information Delivery § The Bureau delivers information via a range of delivery channels. These include: o o o o o Telephone Fax HF Radio Private Networks and GTS Public Web Registered User Forecasters Media representatives FTP and Web Services

GIS 2 Web Basics § GIS 2 Web is the Bureau’s term for delivering Spatial Enabled data via interoperable web services (HTTP) and FTP. § Spatially enabled means the data has been given x, y and/or z coordinates relative to map datums or projections. § Interoperability allows multiple entities (systems/ organisations) to work together directly (Syntactic vs semantic) § Web services refers in this case to Open Geospatial Consortium / OGC web services, in particular WMS and WFS. But in the future it will probably include WCS WPS WSC IWS Net. CDF and OPe. NDAP. § ISO Metadata is delivered via Geo. Network.

Brief History of GIS 2 Web § GIS 2 Web Trial: About 5 years ago the Bureau was involve in a spatial data interoperability trial as was awarded. § GIS 2 Web Experimental: Building on the trail and adding security and user specificity (via SIMS integration) and experimental OPe. NDAP services for research. § GIS 2 Web Operational: This AMDISS system is now operational (both back and front end) and includes the Operational FTP services. § A future developments: Internal GIS 2 Web server (Map. Server and Geo. Server), Operational OPe. NDAP services for registered users and Secure OPe. NDAP services. § The current architecture will likely change as a result of internal developments by both the National Climate Centre and the Water Division.

Bo. M Spatial Data Stack / Data Flow Updated – 2010/06/16 – b. wilson User Communities EM, CT, CI, Marine, Hydrological, Meteorological, Research, GIS etc Bo. M –Web Open. Layers External Clients Internal Clients Spatial Web Services Bureau Symbology/ Standard Catalogues ISO Metadata SIMS Catalogue Arc. Catalog Proj 4 WMS WFS T-WFS DAP HTTP FTP WCS KML FTP Services OGC Web Services Non-OGC Services Spatially Enabled Data for Web Mapping Services and Staging Databases Spatial Enablement libraries OPe. NDAP NCML GDAL ORG Shapelib IDL Primary Data Stores Geospatial and non-spatial CMSS RTDB AIFS DB A to X Hydro Sites. DB FDB MARS Middleware OPe. NDAP SDE ADFD SAM NEONS ADAM File Storage Spatial DB Spatial Source Data and Data Processing Sensor Output GTS AWS Numeric Model Output Marine Satellite Radar Spatial Model Output Spatial Data

OGC and OPe. NDAP / Applications Stacks Web Browsers Spatial Data Admin Analysis/Generation Data Clients Map. Info UDi. G Open. Jump Google Earth Open. Layers Tile Cache Arc. GIS Explorer ESRI Geo-portal Web Portals Geo. Server OPe. NDAP FTP Map. Server IDL Apache/Tomcat Data and Mapping Services Arc. Catalog SIMS Catalogue Proj 4 Catalogue Post. GIS FOSWIKI Apache/Tomcat Catalogues / Documentation ESRI SDE Geo. Network ISO Metadata OGR/GDAL Shape. Lib OPe. NDAP File. Server IDL Direct Access Spatial Enablers Oracle Data Sources Postgre. SQL File Data KML, GML, ESRI Shapefiles, Map. Info, CSV, Geo. Tiff/Tiff, HDF, Net. CDF, Grib , Bufr etc.

GIS 2 Web Project § GIS 2 Web is the Bureau’s Project to deliver Spatially Enable data via OGC web Services and (Open. DAP and FTP). § GIS 2 Web is able to handles Data via GDAL / OGR. o Formats: ESRI Shapefiles, GML, net. CDF, Geo. Tiffs, ACSII § Open Geospatial Consortium (OGC) compliant web services o Web Map Services (WMS)-Version 1. 1. 1 o Web Feature Services (WFS) -Version 1. 0 § Future OGC services will include; o o Web Coverage Services (WCS) for gridded data. Web Catalogue Services (CSW) for data catalogues Web Processing Services (WPS) for client end processing KML services

AWS Process Model

Advantages of Spatial Web Services § Users can integrate Bureau data with their data, Using their preferred desktop or internet mapping/GIS package § Less data management (no data translation, storage, maintenance) § ‘Point of truth’ dataset ie data from the authoritative data source § Up-to-date data – crucial for emergency management § Standard symbology § Eliminates the barriers of data sharing between organisations § Not dependent on proprietary software § More time for analysis § Saves time and money § 24/7 Operational

Standards § GIS 2 Web is made possible through the implementation of spatial interoperability standards including: o Web Map Service (WMS) o Web Feature Service (WFS) o Web Coverage Service (WCS) o Geographic Mark-up Language (GML) o Styled Layer Descriptor (SLD) o WMO Symbology / All Hazards o ISO 19000 series § Other standards include: Net. CDF and HDF, DAP and THREDDS, FTP RFC 2428 - submitted to the Internet Engineering Task Force (IEFT) in 1998

WMS Layers Operationally Available • Radar rainfall intensity (updated every 6 -10 minutes for 60 radar sites) • Mean Sea Level Pressure from ACCESS model (updated 4 times a day and include forecasts from analysis time to 72 hours in hourly time steps) • 10 m Wind Analysis from ACCESS model (updated 4 times a day and include forecasts from analysis time to 72 hours in hourly time steps) • Sea Surface Temperature (updated daily) • Satellite IR – Black and White (updated hourly) • Satellite IR – Colour (updated hourly) • Surface Weather Observations (updated every 15 minutes) • Smoke Dispersion Models (updated twice daily and include forecasts from 00 Z to 48 hours from fixed sourced locations) Victoria only • Tropical Cyclone Track Maps (updated as needed during Tropical Cyclone season) • Bureau boundaries and location data (Radar Locations, Radar Optimum Extent Areas, Coastal Waters, High Seas, Public Weather Forecast Districts, Rainfall Districts and Tropical Cyclone Service Areas)

Radar § Converted from RLE ASCII polar files to point based shapefiles and directly served. Previously served via Post. GIS. § Grouped by State to increase useability. Updated every 10 minutes (6 minutes for Melbourne, Sydney, Brisbane) Emerald and Serpentine radar have come on line in July and Namoi is scheduled for this month. Planned radars include Hobart (TAS), Mt Isa (QLD) and Croker Island/Maningrida (NT) § §

No data for radars § Outages include routine maintenance and unscheduled outages. § Radius is now set 250 km and has “No Data…” Text

Surface Weather Observations § Converted from AXF (ASCII) format § Any new updates published every 15 minutes § Data is accessed from Post. GIS.

Surface Weather Observations § § § § § Station ID and Name YMMB § Cloud cover, visibility, weather conditions Time (UTC) QNH (Pressure) 0130 1017. 1 Rainfall since 9 am 0. 0 last 10 minutes 0. 0 Relative Humidity 35 Dewpoint Temperature 10. 5 Dry bulb Temperature 30. 0 Wind barbs and Maximum Gust last 10 minutes - 30

Numerical Weather Prediction (NWP) § Limited Area Prediction System (LAPS) model has been replace with ACCESS models. § Converted from net. CDF files - Direct reading in future. § Interpolated into isobars for o Mean Sea Level Pressure (MSLP) § Points and raster for o Wind Analysis (10 m) § Geo. Tiff o Sea Surface Temperature – daily

ACCESS Model § The ACCESS Model was launch on 18 August. § It is running on the Bureau’s new supercomputer. § Based on the UK Model § Is more accurate than the GASP and LAPS Models. § Experimental Web Services

MSLP § Mean Sea Level Press has been improved with the new ACCESS model. § MSLP and Winds updated four times daily § Include forecasts from analysis time to 72 hours in 3 hourly time steps

Winds (barbs and wind speed) § Wind barbs require the use of Post. GIS. § Mean Sea Level Press has been improved with the new ACCESS model. § MSLP and Winds updated four times daily § Include forecasts from analysis time to 72 hours in 3 hourly time steps

Sea Surface Temperature §Sea Surface Temperature updated daily §The resolution of the output grid is 0. 25 degrees. §http: //reg. bom. gov. au/reguser/b y_prod/metadata/sst. xml

Satellite IR (Black and White) § Updated hourly § MTSAT-IR / JMA

Satellite IR (Colour) § Updated hourly § MTSATIR / JMA

Smoke Dispersion Forecast Plumes § Based on HYSPLIT model § Tool for prescribed burns § Forecast each hour from 10 – 23: 00 hr for current and next day. § Victoria only

Tropical Cyclones § Track Maps developed in application called TC Module § Data saved as GML § Data translated to GML simple feature and shapefiles

Tropical Cyclones § Fix Points § Track Lines § Wind Areas § Forecast of Track Areas § Warning Areas § Watch Areas

BOM Boundary and Site Layers § Boundary Layers o Radar Extent Areas o Radar Locations and Type o Coastal Waters, High Seas and Public Weather Forecast Districts o Rainfall Districts

Radar Extents and Sites

BOM Boundaries § Caching is available for static information

WFS Layers Operationally Available § Surface Weather Observations (updated every 15 minutes) - Current § Surface Weather Observations (updated every 15 minutes) for last 24 hours § Tropical Cyclone Track Maps (updated as needed during Tropical Cyclone season)

Grouping and Descriptions § Due to the complexity and number of service themes the Bureau has adopted Plain English descriptions and hierarchical grouping to enhance useability.

ISO 19115 Metadata § Available on web at http: //www. bom. gov. au/reguser/by_prod/metadata/ § Also within the Get. Capabilities of document <Metadata. URL type="TC 211"> <Format>text/xml</Format> <Online. Resource xmlns: xlink="http: //www. w 3. org/1999/xlink" xlink: type="simple" xlink: href="http: //www. bom. gov. au/reguser/by_prod/metadata/cwaters. xml" /> </Metadata. URL>

Metadata on the Web

Geo. Network § Have been working with Blue. Net MEST since 2007. § Have tested this on harvesting records from DWD and CMA. § We are working on an Operational External and Internal MEST. § Large number of profile types in the Bureau

Accessing GIS 2 Web Operational System § Need to be a registered user § Contact Registered Users Services Manager p. george@bom. gov. au to determine status or to register § Usage charges for non-emergency services agencies from 1 July 2009 § An evaluation account is available for one month § FAQ available at http: //www. bom. gov. au/faq/ or http: //www. bom. gov. au/catalogue/

Accessing GIS 2 Web Operational System § Details stored in Bureau’s Services Information Management System (SIMS) § Provides means for authorisation and authentication o Allows for IP authentication for clients that do not support HTTP authentication

SIMS is the products and user catalogue products and users. It is used to generate the 100 s of web services and to automate web page development. It is an in-house Oracle Development.

Future layers § Flood Warnings § TITAN Thunderstorm Identification, Tracking, Analysis and Nowcasting § Aviations Products § Radar Rainfields § Nex. GEN § ACCESS § Climate Data ( http: //www. bom. gov. au/climate/data/ )

Hydrology § River conditions o Water Level o Tendency § Update every 15 minutes § Looking at other states.

Rainfall Estimates § § Melbourne, Sydney, Brisbane, Adelaide § 10 min accumulations radar only and updated every 6 or 10 minutes since 9 AM, daily accumulations blended with rain gauges and updated every 30 min

Rainfall Forecasts 60 min accumulation Probability of rain > 50 mm Forecast time series at a point with uncertainty shown 0 -90 Minute § Capital city forecasts, 1 km & 6 min resolution, 250 km domain § Regional forecasts, 2 km & 10 min resolution, 500 km domain § 30 member ensemble updated every 6, 10 minutes § 30, 60, 90 min accumulations of ensemble mean (expected rain) § Probability that rain accumulation will exceed 1, 2, 5, 10, 20, 50 mm in next 60 minutes 1 -6 Hour Forecasts

Aviation Products § Terminal Aerodrome Forecast (TAF) § Trend Type Forecast (TTF) § Area Forecast (ARFOR) § SIGMET § Significant Weather Charts (SIGWX)

ACCESS Model Direct reading of ACCESS net. CDF files Will be tested on the Experimental Service The experimental service is being virtulised and upgraded next quarter.

Questions b. wilson@bom. gov. au