Intelligent integration for nowcasting Selected slides from a

Intelligent integration for nowcasting Selected slides from a talk given at the 38 th Annual Congress of the Canadian Meteorological and Oceanographic Society. For the complete powerpoint file see: A Fuzzy Logic-based Analog Forecasting System for Ceiling and Visibility, 38 th Annual Congress of the Canadian Meteorological and Oceanographic Society, May 31 -June 3, 2004, Edmonton, Alberta. http: //arxt 39. cmc. ec. gc. ca/~armabha/papers_and_presentations

Future Role of Operational Meteorology Scientific and systematic forecast process Partnership with technology How?

Intelligent Weather Systems (RAP/NCAR) 1 Weather Radar Nowcasts RAP, Thunderstorm Auto-Nowcasting, www. rap. ucar. edu/projects/nowcast Sensor Systems Real-Time Data Preprocessing Human Input (> 15 min) Fuzzy Logic Integration Algorithm Quality Control Data Assimilation Mesoscale Model IWS Design GUI Real-Time Data Algorithms Product Generator Model Output Algorithms • Expert system development framework • Applies existing knowledge, techniques and algorithms • Achieves intelligent integration of all relevant, real-time data Selective Climatological Input • Supports rapid development of useful, maintainable operational applications 1. RAP, Intelligent Weather Systems, www. rap. ucar. edu/technology/iws/design. htm User

Intelligent Weather Systems (RAP/NCAR) 1 Fuzzy logic integration algorithm Human input For example, a fuzzy rule forecasting radiation fog: 2 Decision If sky clear and wind light and humidity high and humidity increasing For example, choice of Then chance of radiation fog is high data and fcst technique Fuzzy Rule Base Satellite image W 1 low med hi Wind speed low W 2 med hi Humidity Matrix of fuzzy rules covers space of all predictors Humidity trend System can run continuously to give real-time, smart forecast quality control. For details, see examples. 3 Chance of radiation fog (qualitative description) 1. RAP, Intelligent Weather Systems, www. rap. ucar. edu/technology/iws/design. htm 2. Jim Murtha, 1995: Applications of fuzzy logic in operational meteorology, Scientific Services and Professional Development Newsletter, Canadian Forces Weather Service, 42 -54 3. Meteorological applications of fuzzy, http: //chebucto. ca/Science/AIMET/applications

Operational Meteorology A Scientific and Systematic Forecast Process: a partnership with technology! 1 Technology Observation Sat, radar, awos… Analyses 4 DVAR, AI… Diagnoses RDP, AI… Prognoses GEM, EPS, UMOS… Products/ Services SCRIBE/AVIPADS, etc. W O R K S T A T I O N Meteorologist Reports from public Pattern recognition Conceptual models Science, experience, training Decisions Performance Measures 1. Jim Abraham, 2004: Science-Operations Connection workshop, Meteorological Service of Canada, Toronto, 24 -26 February 2004.

“Smart Alert” Concept Impending problem Bust

St. John’s Fit Loose Tight | | l | | | l l| | | | | l | |l| | | | |l | l | | | … | | l l| | | Wind 00 h 1215 01 h 1314 02 h 1412. . . 12 h 1408 Ceiling Visibility Direction Speed Time … Weather 00 h R-L 01 h R-L 02 h L. . . 12 h L- Search Make Save Send 100+ 60 30 25 20 15 10 9 8 7 6 5 4 3 2 1 21 22 23 0 1 2 3 4 5 6 7 8 9 10 11 12 AMD TAF CYYT 270010 Z 270024 1315 KT 2 SM -RA BR OVC 006 TEMPO 0002 1/2 SM -DZ FG OVC 003 FM 0200 Z 14010 KT 1/2 SM -DZ FG OVC 002 TEMPO 0224 1/4 SM -DZ FG OVC 001 RMK NXT FCST BY 06 Z=

DECISION SUPPORT SYSTEMS * official forecast Battleboard ! actual trend 0 time ACTUAL WEATHER MAP (animated) HEADS-UP ALERT & DISPLAY GUI leverages FORECASTER forecaster’s actions raises forecaster’s situational awareness (interacts, intervenes) awareness and knowledge Graphic intervention First resort MODELLED WEATHER MAP (editable) GUIDANCE DISPLAY (satellite, NWP, etc. ) Direct intervention Last resort DSS (interaction with integration and prediction) PRODUCT DISPLAY (editable) POSTPROCESSING DA RADAR NWP data METAR data REAL-TIME OBS SATELLITE PRODUCTS MODEL-BASED WEATHER ELEMENTS information FORECAST INTEGRATION data PRODUCT GENERATION UPPER AIR EXTRAPOLATION AI knowledge PROJECTED OBS CONSISTENCY CHECKING CLIMATE ARCHIVE data TRANSLATION RAW, QC’d WEATHER PRODUCT SPECIFICATIONS data PREDICTION MODELLED WEATHER data and information • up-to-the-minute intelligent data fusion • abstract features • derived fields • intelligently composed “interest fields” USER information • special interests • cost-based decision-making models VERIFICATION * Forecaster Workstation User Requirements Working Group meeting notes, 2000: Decision support systems for weather forecasting based on modular design, updated slightly for Aviation Tools Workshop in 2003.

Decision Support Systems Design Generic: no-name, conceptual design that could link and integrate the most useful elements of WIND, AVISA, Multi. Alert, SCRIBE, FPA, URP, and so on in evolving WSP application, Nin. Jo. Modular: shows where distinct sub-tools / agents can be developed. Working in this way, individual developers could work on isolated sub-problems and anticipate how to plug their results into a larger shared system. As technology inevitably improves, improved modules can be easily installed and quickly implemented. User-centered: forecast decision support systems from forecaster's point of view, designed to increase situational awareness. Hybrid: combines complementary sources of knowledge, forecasters and AI, to increase the quality of input data and output information. Intelligent integration of data, information, and model output, and use of adaptive forecasting strategies are intrinsic in this design.

Hybrid Forecast Decision Support Systems Hybrid forecast system development is a current direction of the Aviation Weather Research Program (AWRP) 1 and the Research Applications Program (RAP), 2 NCAR (the main organizers of AWRP R&D). “If a statistical / analog forecast disagrees with a model forecast, or if different sensors disagree about how C&V are measured, what should we do about it? Fuzzy logic could simulate how humans might apply confidence factors to different pieces of information in different scenarios. ” 3 AWRP Terminal Ceiling and Visibility Product Development Team (PDT) project, Consensus Forecast System, a combination of: · COBEL, a physical column model 4 · Statistical forecast models, local and regional · Satellite statistical forecast model 1. Aviation Weather Research Program, http: //www. faa. gov/aua/awr 2. Research Applications Program, http: //www. rap. ucar. edu 3. Norbert Driedger, 2004, personal communication. 4. Cobel, 1 -D model, http: //www. rap. ucar. edu/staff/tardif/COBEL

Hybrid Forecast Decision Support Systems AWRP National Ceiling and Visibility PDT research initiatives: 1 · Data fusion: intelligent integration of output of various models, observational data, and forecaster input using fuzzy logic 2, 3 · Data mining, C 5. 0 pattern recognition software for generating decision trees based on data mining, freeware by Ross Quinlan (http: //www. rulequest. com), like CART · Analog forecasting using Euclidean distance development of daily climatology for 1500+ continental US (CONUS) sites · Incorporate Auto. Nowcast of weather radar in 2004 -2005 4 · Incorporate satellite image cloud-type classification algorithms 5 1. Gerry Wiener, personal communication, July 2003. 2. Intelligent Weather Systems, RAP, NCAR, http: //www. rap. ucar. edu/technology/iws 3. Shel Gerding and William Myers, 2003: Adaptive data fusion of meteorological forecast modules, 3 rd Conference on Artificial Intelligence Applications to Environmental Science, AMS. 4. Auto. Nowcast, http: //www. rap. ucar. edu/projects/nowcast 5. Tag, Paul M. , Bankert, Richard L. , Brody, L. Robin. 2000: An AVHRR Multiple Cloud. Type Classification Package. Journal of Applied Meteorology: Vol. 39, No. 2, pp. 125 -134.

Hybrid Forecast Decision Support Systems 1. Herzegh, P. H. , Bankert, R. L. , Hansen, B. K. , Tryhane, M. , and Wiener, G. , 2004: Recent progress in the development of automated analysis and forecast products for ceiling and visibility conditions, 20 th Conference on Interactive Information and Processing Systems, American Meteorological Society.

Fuzzy Logic at Research Applications Program, NCAR According to Richard Wagoner, Deputy Director at Research Applications Program (“Technology Transfer Program”), NCAR: 1 • NCAR / RAP is now a “continuous set theory” [fuzzy set theory] development center. • Over 90% of systems developed use fuzzy logic [FL] as the intelligence integrator. [ … P. S. It is now 100% 2 ] • [FL offers] unprecedented fidelity and accuracy in systems development. • Automatic FL-based systems now compete with human forecasts. 1. Richard Wagoner, 2001: Background briefing on post processing data fusion technology at NCAR, online presentation, http: //www. rap. ucar. edu/general/press/presentations/wagoner_21 feb 2001. pdf 2. John K. Williams, 2004: Introduction to Fuzzy Logic as Used in the NCAR Research Applications Program, Artificial Intelligence Methods in Atmospheric and Oceanic Sciences: Neural Networks, Fuzzy Logic, and Genetic Algorithms, Short Course, American Meteorological Society, 10 -11 January 2004, Seattle, WA. ftp: //ftp. rap. ucar. edu/pub/AMS_AI_Short. Course/Williams_AMS_Short. Course_11 Jan 2004. pdf

Fuzzy logic Since we can assign numeric values to linguistic expressions, it follows that we can also combine such expressions into rules and evaluate them mathematically. A typical fuzzy logic rule might be: If temperature is warm and pressure is low then set heat to high A graphical illustration to fuzzy logic, http: //www. mcu. motsps. com/lit/tutor/fuzzy. html

How Rules Relate to a Control Surface A fuzzy associative matrix (FAM) can be helpful to be sure you are not missing any important rules in your system. Figure shows a FAM for a control system with two inputs, each having three labels. Inside each box you write a label of the system output. In this system there are nine possible rules corresponding to the nine boxes in the FAM. The highlighted box corresponds to the rule: If temperature is warm and pressure is low then set heat to high A graphical illustration to fuzzy logic, http: //www. mcu. motsps. com/lit/tutor/fuzzy. html

Three Dimensional Control Surface The input to output relationship is precise and constant. Many engineers were initially unwilling to embrace fuzzy logic because of a misconception that the results were not repeatable and approximate. The term fuzzy actually refers to the gradual transitions at set boundaries from false to true. A graphical illustration to fuzzy logic, http: //www. mcu. motsps. com/lit/tutor/fuzzy. html

Intelligent integration for nowcasting For more information, see: A Fuzzy Logic-based Analog Forecasting System for Ceiling and Visibility, 38 th Annual Congress of the Canadian Meteorological and Oceanographic Society, May 31 -June 3, 2004, Edmonton, Alberta. http: //arxt 39. cmc. ec. gc. ca/~armabha/papers_and_presentations