The CARDS System Description and Algorithms CAnadian Radar

The CARDS System Description and Algorithms CAnadian Radar Decision Support Paul Joe Meteorological Service of Canada

Outline • • • Introduction Requirements / Issues The CARDS Solution Algorithms, Products, Functionality Example of Usage

Introduction TITAN = Thunderstorm Initiation, Analysis and Nowcasting (NCAR “free*”) WDSS II = Warning Decision Support System (NSSL “free*” ) CARDS = Canadian Radar Decision Support (EC “free**”) • *Download from web • ** Discuss

Introduction • Operational system of the Meteorological Service of Canada • Single radar processing systems for multiple uses • In transition, being integrated with forecaster workstation (Nin. Jo)

The Requirements

The Severe Warning Challenge • Specificity of information is needed to be effective – Time/duration, Location, Type of Event • • • Distinguish between severe and non-severe, And tornadic and non-tornadic thunderstorms. Looking for the rare event, many types of severe storms Large forecast area Work Load, Efficiency 3, 000 km 2

High resolution composites

Yellow and white = events Green = thunderstorms The Rare Event 100 km Thunderstorm locations and reported severe weather

High Level Requirements An expert can… • Recognise patterns • Detect anomalies • Keep the big picture (situational awareness) • Understand the way things work • Relate past, present, and future events • Pick up on very subtle differences • Observe opportunities, able to improvise • Address their own limitations The system design must enable this!

Situational Awareness

The Canadian Warning Offices > 3, 000 square km per forecast office

Screen Real-estate Issue Poor Efficiency

Supporting Mental Models

Using Algorithm Approch An algorithm searches the data for relevant patterns (spatial or temporal). • Not an automated answer! • Individual algorithms are configured to have high POD –but results in high FAR • Combination of algorithms: –support each other to reduce the FAR –create leverage points for further inquiry –support use of the conceptual model –support expert decision-making

Enabling Expertise • Can not do anything if only the answer is provided! – This will make anyone dumb! – Self-fulfilling prophesy • Must be able to “access or drill down” to the underlying data

Functionality

Recall Manual Analysis Process. . … We want to mimic this – but quickly • • High reflectivity Echo top Shapes Gradients of reflectivity • Trends • Movement • Flair echo/Hail in dualpol • Relationships – Updraft Tilt – Weak Echo Regions (WER) – Bounded WER – Location – Echotop - Gradient • Rotation • Divergence • Convergence

Data Access

Cell View to access to data/products Cell View Echo Top hail CAPPI’s Automated XSECT gradient VIL Time history

Animation to show the functionality and use of cell views

Algorithms Approach • Not the answer! but … • Create “Leverage” Points • Support your Conceptual Model • Support Decision Making

Algorithm • A set of computer procedures or steps • Attempts to match human visual/pattern recognition skills • Software that identifies a feature in the data that represents a meteorological feature (e. g. , a thunderstorm cell, a cell track)

Products/Algorithms (configurable) • • • • CAPPI (many) MAXR Height of MAXR Echo. Top VIL, Downdraft, Hail Size Reflectivity Gradient PPI’s Radial Velocity Spectral Width Corrected Reflectivity Precipitation Accumulations Composites of various products Interactive Cross-sections Algorithm Ensemble Product Cell Views Storm Cell Identification Table • Cell Identification – average and max value – locations • • • Bounded Weak Echo Region Mesocylone, downburst, gust Cell Properties – Echotop, VIL, Hail Size – See Product List • • Automatic Cross-sections Tracking, Simple Nowcast Multi-radar algorithm merge Rank Weight – Color Coding • • Sorted Rank Cross-correlation Tracking – Point Forecast

Need for “Leverage” Points Algorithms Where is the rotation/Tornado Vortex Signature? Leverage = “look at me”

2356 UTC

It is also about relationships!

Forecasters need to maintain situational awareness: #1 problem of missed warnings but which cell is the dangerous one? NO NEED FOR SINGLE RADAR PRODUCTS! But…

Forecasters must be able to diagnose the salient features to make a warning decision Severe Storm Features -Large cell with strong elevated reflectivity (MAXR>45 d. BZ) -Tall (high echo top) -Hail -Low level Reflectivity gradients under highest echo tops -Weak Echo Region -Hook/Kidney beam shape -Mesocyclones -Downdrafts Codifying the Lemon Technique through Cell Views

Some of the Algorithms • Hail • Downdraft Algorithm • Storm Classification Identification and Tracking • Ranking Storms

Products/Algorithms (configurable) • • • • CAPPI (many) MAXR Echo. Top VIL, WDraft, Hail Size Reflectivity Gradient PPI’s Radial Velocity Spectral Width Corrected Reflectivity Precipitation Accumulations Composites of various products Interactive Cross-sections Algorithm Ensemble Product Cell Views Storm Cell Identification Table • Cell Identification – average and max value – locations • • • Bounded Weak Echo Region Mesocylone, downburst, gust Cell Properties – Echotop, VIL, Hail Size – See Product List • • Automatic Cross-sections Tracking, Simple Nowcast Multi-radar algorithm merge Rank Weight – Color Coding • • Sorted Rank Cross-correlation Tracking – Point Forecast

The Hail Algorithms Hail Shaft

S 2 K Hail Products • Polarimetric, BOM/MSC, WDSS • BOM/Treloar Empirical Algorithm – Uses height of 50 d. BZ echo, VIL and freezing level • WDSS – Uses height diff of freezing level and 45 d. BZ top, VIL, hail kinetic energy (fn of d. BZ), temperature profile – Probability of severe hail – SHI

Hail Size, VIL & Freezing Level

Hail Size, Height of 50 d. BZ echo and Freezing Level

Hail Product (Image and Feature)

WDSS HDA Probability of Hail (POH) • Estimate the probability of any size hail associated with a storm • H 45 = Height of the 45 d. BZ echo AGL (km) • H 0 = Height of the melting level AGL (km) -> Δ H Based on data from a Swiss hail suppression experiment

HDA Severe Hail Index (SHI) • Vertically Integrated Liquid (VIL) (Emphasis given to lower d. BZ) – To remove “hail contamination” • Hailfall Kinetic Energy (E) (Emphasis given to higher d. BZ and those d. BZ above the melting layer) E = 5 x 10 -6 x 100. 084 Z x W(Z) – W(Z) = 0 for Z < 40 d. BZ – W(Z) linearly interpolated for 40 d. BZ > 50 d. BZ – W(Z) = 1 for Z > 50 d. BZ

HDA Severe Hail Index (SHI) • Weighted by thermodynamic profile – Obtained manually from nearby sounding, or – Obtained automatically from mesoscale model analysis • Greater temporal and spatial resolution N • • SHI = 0. 1 W (H ) E H T i i i Prob. Of Severe Hail (POSH; dia > 1. 9 cm) and Max. Estimated Hail Size (MEHS) derived from SHI (Witt et al. 1998) i WT(H)

Hail algorithm Hail Shaft

S 2 K Comparison Average Hail Size POL • • Uses height of 50 d. BZ echo, VIL and freezing level Max WDSS – – – • WDSS Polarimetric, BOM/MSC, WDSS CARDS/BOM/Treloar Empirical Algorithm – • CARDS Uses height diff of freezing level and 45 d. BZ top, VIL, hail kinetic energy (fn of d. BZ), temperature profile Probability of severe hail SHI What is the truth? Do you want to just reduce the CSI or do you want high POD? What is the relationship to your forecast product? Ave OBS Obs

Comparison Average Hail Size C Band Dual Pol S Band CARDS C Band CARDS S Band WDSS OBS

PDF of Hail Size

CARDS Hail Size Time Sequence Nov 3 Case MAX Ave Harold Brooks

WDSS Probability of Hail obs any sever Harold Brooks

WDSS Max Hail Size Harold Brooks

The Downburst/Gust Potential Algorithm

WDraft Product Speed of the outflow • • Theoretically based on work by Emmanuel Empirically adapted by Stewart, OU VIL -> downdraft strength -> outflow strength Earlier warning than just the surface divergence product • Uses volume scan reflectivity data

Gust Potential Algorithm • Outflow Speed W = (20. 63 VIL – 3. 125 x 10 -6 H 2 )1/2 W = outflow speed m/s H = Echotop height • Threshold = 10 m/s

Example TIME Increasing Reflectivity Based Surface Doppler Indication of Strong Gust Courtesy of Isztar Zawadzki Microburst from radial velocity at surface

Storm Classification Identification and Tracking Ranking

Cell Ranking • Objective: – find the most dangerous and strongest storm – Reduce FAR of individual high POD algorithms • Algorithm: use cell properties to compute a single metric – rank weight • Sort the rank weights to find the strongest storm

WDSS Ranking Table Rankings: Rank = Circulation(f*109) + (10*Size + 10*POSH)*105 + 10*Damaging Wind Index

WDSS Ranking Results (Nov 3 Case) Ranked High

CARDS Cell Analysis Summary Storm Classification Identification Table Storm Number Rank (order) Rank Wt (severity) Category (X) Down. Draft (m/s) BWER (ht) Meso Shear Rank Wt = ∑ αi vi Hail Size VIL/VIL density Max db. Z Echo. Top Ht Speed

Rank Weight • A parameter to numerically summarize the various attributes of the cell object Rank Wt = ∑ αi vi • α is an empirical coefficient that normalizes and scales the parameter v by severity • Normalization done by categorizing the parameter

Storm Rank Weight Each parameter is categorized on a scale from 0 to 4 (normalizing) Rank Weight is the average of the categorized values. Parameters are configurable. Used to determine a numeical value for sorting.

CARDS Supporting the Mental Model

Ensemble/Algorithm View Supporting the Mental Model Lemon/Doswell

Automatic Vertical Cross-section

More? ? ? G 96

System Design

Network Topology

The System Design

Cell Processing

Computer Hardware • Server – S 2 K - Single dual processor, 600 Mhz, 1 Gbyte RAM, 2 x 18 Gbyte Hard drive, Linux PC – MSC – Linux Cluster, central node for data ingest and science processing, secondary nodes for product/image creation • Client – S 2 K/MSC - PC to run Netscape or Java Application for the “Interactive Viewer” to access and display the products

Region Growing Algorithm

Generic Approach • Conceptual Model • Data (2 D or 3 D, 1 D or mx. D) • Translate Conceptual Model to a Data/Sensor Model • Define an Interest Field • Define a detection threshold • Search for elements exceeding threshold, grow in the various dimenstions

Pattern Recognition Algorithm Glossary 1 • “Interest” Field – A grid of data of a parameter related to the “object” – 2 D or 3 D or … – Polar or cartesian or … • Pattern Vector Element – a single grid point that exceeds a “threshold value” • Pattern Vector – a contiguous line of pattern vector elements exceeding a threshold value – 1 dimensional • 2 D Feature – a contiguous set of pattern vectors - 2 dimensional • Height Associated Feature (or 3 D Feature) – a set of 2 D features at different heights (in practice) • Time Associated Feature (or 4 D Feature) – a “tracked” Height Associated Feature

Glossary 2 • Weather Object – a Feature that satisfies constraints, rules, filters, classifications, thresholds, etc – interpreted as a possible meteorological concept – Eg cell, mesocyclone, microburst, area of hail, area of lightning • Storm or Cell Attribute – an property of a storm – e. g. average value, max value, area, % of positive strikes, etc

Glossary 3 • Field – a two dimensional array of a (radar or derived) parameter – eg PPI of reflectivities, echotop heights • Template – a two dimensional area defined by the extents of the pattern vectors of a feature – Subset of a field

Cell or Feature Identification Example of the Approach

Basic Approach e. g. Thunderstorm Identification • Thunderstorm = cell • Cell definition – contiguous area of reflectivity (the Interest Field) above a certain threshold – Could be from a PPI, a CAPPI, MAXR or VIL, lightning, hail from polarimetric radar, satellite or … • Define Threshold • Objective: – – Identify individual cells Determine their location Determine the footprint (perimeter) Compute properties

Interest Fields and Thresholds Summary

Interest Fields • • Cells – CAPPI, MAXR, VIL Mesocyclone – azimuthal shear Microburst – radial shear Hail – hail size field

Thresholds • Cells – fixed (45 d. BZ), multiple (25, 30, 35, 40 d. BZ), adaptable (displacement from peak) • Mesocyclones – 2 m/s/km, -2 m/s/km • Microbursts - -2 m/s/km • Hail – 0. 1 cm

Detection/Classify • Try to find all potential features (rare events) – High probability of detection • Reduce high false alarms – Consistency with other features – Use other properties, shape, location – Forecaster

The Reflectivity Threshold CARDS = 45 d. BZ WDSS = 45 d. BZ BOM =35 and 45 20 d. BZ 30 d. BZ 40 d. BZ

Multiple Threshold Technique (WDSS II/NSSL)

Peak Detection Technique (Meteo-France/Meteo. Swiss)

The Interest Field MAX R = 2 D Projection of 3 D Data CAPPI MAXR 9. 0 km CARDS 45 d. BZ 30 d. BZ 7. 0 km 3. 0 km 1. 5 km WDSS ii

The Region Growing Algorithm

Radar Data in Polar Coords • The “interest field” • Radar Co-ords

Region Growing Algorithm • Find contiguous pixels/bins of data that exceed a threshold • Terminology – Element -> pattern vector -> 2 D feature – Pixels -> line of pixels -> pixel areas

PV Elements 45 d. BZ Threshold Reflectivity = 45 d. BZ

Pattern Vectors

Feature (2 D) Cell = Feature = Group of Pattern Vectors MAXR field + + • Specify a d. BZ threshold • Find Pattern Vectors • Collate PV’s into a Feature

Feature (3 D) – Height Association! + +

Cell Properties • Use footprint defined by cell identification on MAXR or VIL or … • Use another interest field and find max, average and their locations – E. g. echotop, wdraft, hail, etc • Can then plot the locations or use to automatically determine the cross-section points.

Summary • Brief description of the CARDS system • Setup for high probability of detection, results in high false alarm rate • Use the combination of algorithm outputs to determine the most intense storms. • Fuzzy logic storm ranking • Rapid access to products • Assume expert user – Maintain situational awareness – Provide guidance/leverage products, drill down to data, hint at where and what to look for in detail, require forecaster for final decision making