Nowcasting of thunderstorms Valliappa Lakshmanan noaa gov National Severe

Nowcasting of thunderstorms Valliappa. Lakshmanan@noaa. gov National Severe Storms Laboratory & University of Oklahoma Seminar at City University of New York CREST program http: //cimms. ou. edu/~lakshman/ Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 1

What is nowcasting? l l Skilled short-term estimates and predictions l Typically 0 -60 minutes l For emergency managers, transportation, etc. l Made by meteorologists l With guidance from automated algorithms Guidance to forecasters involves supplying estimates & predictions for: l Spatial location of thunderstorms l Where is the storm now? What is the path the storm has traveled? l Where will the storm be in 30 minutes? l Intensity of thunderstorms l Weakening? Strengthening? l Potential hazards l Hail? Lightning? Tornadoes? Flooding? Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 2

Hazard prediction l l This talk will focus on estimating and predicting: l Spatial location of thunderstorms l Intensity characteristics of thunderstorms Hazard prediction is carried out by tailored algorithms l Hail Detection Algorithm l Looks for high radar reflectivity aloft l Cores may descend to cause hail l Flash flood prediction algorithm l Estimate rainfall amount based on radar reflectivity l Accumulate rainfall in delineated basins l Couple with flow model (soil moisture, etc. ) l Etc. Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 3

How to do nowcasting l l l Numerical models l Can not be done in real-time l Skill of numerical models an area of much research l May be the future Rule-based prediction of growth and decay l Identify boundaries from multiple sensors or human input l Extrapolate echoes likely to persist or form l Approach of “Auto Nowcaster” from NCAR l Qualitatively: works l Quantitatively: similar issues as numerical models Linear extrapolation of radar echoes l Highly skilled in the short term (under 60 minutes) l Can be done in real-time l Assumption is of steady-state (no growth/decay) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 4

Methods for estimating movement l Linear extrapolation involves: l l l Estimating movement Extrapolating based on movement Techniques: 1. 2. 3. Object identification and tracking l Find cells and track them Optical flow techniques l Find optimal motion between rectangular subgrids at different times Hybrid technique l Find cells and find optimal motion between cell and previous image Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 5

Some object-based methods l Storm cell identification and tracking (SCIT) l Developed at NSSL, now operational on NEXRAD l Allows trends of thunderstorm properties l Johnson J. T. , P. L. Mac. Keen, A. Witt, E. D. Mitchell, G. J. Stumpf, M. D. Eilts, and K. W. Thomas, 1998: The Storm Cell Identification and Tracking Algorithm: An enhanced WSR 88 D algorithm. Weather & Forecasting, 13, 263– 276. Multi-radar version part of WDSS-II Thunderstorm Identification, Tracking, Analysis, and Nowcasting (TITAN) l Developed at NCAR, part of Autonowcaster l l l Dixon M. J. , and G. Weiner, 1993: TITAN: Thunderstorm Identification, Tracking, Analysis, and Nowcasting—A radar-based methodology. J. Atmos. Oceanic Technol. , 10, 785– 797 Optimization procedure to associate cells from successive time periods Satellite-based MCS-tracking methods l Association is based on overlap between MCS at different times l l l Morel C. and S. Senesi, 2002: A climatology of mesoscale convective systems over Europe using satellite infrared imagery. I: Methodology. Q. J. Royal Meteo. Soc. , 128, 1953 -1971 http: //www. ssec. wisc. edu/~rabin/hpcc/storm_tracker. html MCSs are large, so overlap-based methods work well Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 6

Object-based methods, pros & cons l How object-based methods work: l l l Pros: l l l Identify high-intensity clump of pixels as “cells” Associate cells between time frames l Closest distance/values/overlap, etc. Small-scale prediction Can find out history of a thunderstorm (“trends”) Cons: l l Splits and merges hard to keep track of Hard to avoid association errors Most storm cells last only about 20 minutes Large-scale predictions are difficult to build up Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 7

Optical flow methods l How optical flow methods work l l l Do not identify and associate cells l l l Pro: Removes cell identification and association errors Con: No trends possible Not affected by splits/merges l l l Take rectangular region around each pixel of current image Move rectangular window around previous image Choose movement that minimizes error between images Need to ensure that successive pixels do not have very different movements Pro: More accurate motion estimates Con: Small-scale tracking not possible Poor motion estimates where no storms available in current/previous image l l Often have to use global movement Or interpolate between storms Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 8

Some optical flow methods l TREC l Minimize mean square error within subgrids between images l No global motion vector, so can be used in hurricane tracking l Results in a very chaotic wind field in other situations l l Large-scale “growth and decay” tracker l MIT/Lincoln Lab, used in airport weather tracking l Smooth the images with large elliptical filter, limit deviation from global vector l Not usable at small scales or for hurricanes l l Tuttle, J. , and R. Gall, 1999: A single-radar technique for estimating the winds in tropical cyclones. Bull. Amer. Meteor. Soc. , 80, 653 -668 Wolfson, M. M. , Forman, B. E. , Hallowell, R. G. , and M. P. Moore (1999): The Growth and Decay Storm Tracker, 8 th Conference on Aviation, Range, and Aerospace Meteorology, Dallas, TX, p 58 -62 Mc. Gill Algorithm of Precipitation by Lagrangian Extrapolation (MAPLE) l Variational optimization instead of a global motion vector l Tracking for large scales only, but permits hurricanes and smooth fields l Oct. 23, 2006 Germann, U. and I. Zawadski, 2002: Scale-dependence of the predictability of precipitation from continental radar images. Part I: Description of methodology. Mon. Wea. Rev. , 130, 2859 -2873 Valliappa. Lakshmanan@noaa. gov 9

Need for hybrid technique l l Need an algorithm that is capable of l Tracking multiple scales: from storm cells to squall lines l Storm cells possible with SCIT (object-identification method) l Squall lines possible with LL tracker (elliptical filters + optical flow) l Providing trend information l Surveys indicate: most useful guidance information provided by SCIT l Estimating movement accurately l Like MAPLE How? Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 10

Technique 1. 2. 3. 4. 5. 6. Identify storm cells based on reflectivity and its “texture” Merge storm cells into larger scale entities Estimate storm motion for each entity by comparing the entity with the previous image’s pixels Interpolate spatially between the entities Smooth motion estimates in time Use motion vectors to make forecasts Oct. 23, 2006 Courtesy: Yang et. al (2006) Valliappa. Lakshmanan@noaa. gov 11

Why it works l Oct. 23, 2006 Hierarchical clustering sidesteps problems inherent in object-identification and optical-flow based methods Valliappa. Lakshmanan@noaa. gov 12

Advantages of technique l Identify storms at multiple scales l l l No storm-cell association errors l l Use optical flow to estimate motion Increased accuracy l l l Hierarchical texture segmentation using K-Means clustering Yields nested partitions (storm cells inside squall lines) Instead of rectangular sub-grids, minimize error within storm cell Single movement for each cell Chaotic windfields avoided l l l No global vector Cressman interpolation between cells to fill out areas spatially Kalman filter at each pixel to smooth out estimates temporally Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 13

1. Identifying storms: K-Means clustering l l l Obtain a vector of measurements at each pixel l Statistics in neighborhood of each pixel (called “texture”) l Can also use multiple sensors or channels Divide up vector space into K “bands” l The bands can be equally spaced by equal-probability l Center the clustering algorithm at each of these bands l Assign each pixel to the band that it lies in Perform region growing l Pixels in same band adjacent to each other are part of region l Compute region properties Move pixel from one region to another if cost function lowered l Cost function lower if pixel moves to region whose mean texture it is closer to l Cost function lower if pixel moves to region that it is closer (spatially) to Iterate until stable Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 14

The cost function l The cost function takes into account l Textural similarity between pixel at x, y and the mean texture of kth cluster l Spatial contiguity of pixel to cluster l Weighted appropriately (lambda=0. 2 seems to work well) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 15

Clustering: example Radar reflectivity Oct. 23, 2006 K=4 clustering Valliappa. Lakshmanan@noaa. gov 16

2. Hierarchical clustering l At the end of iteration, all pixels have been assigned to their best clusters l l Combine clusters to form larger regions l l l Most detailed scale of segmentation Scale=0 Clusters are typically very small Find mean inter-cluster distance Combine regions which are spatially adjacent whose textural means are close to each other Reflectivity Scale=0 Repeat to get largest regions Scale=1 Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov Scale=2 17

3. Compute motion estimates l l l Starting with scale=2, project the current cluster backward l Move the cluster around within the previous image l Choose the movement that minimizes mean absolute error l Minimization based on kernel estimate, to reduce outlier errors A motion estimate obtained for each cluster l Less noisy than pixel-based estimates l Automatic smoothing over region of cluster l Scale=0 is the noisiest (fewer pixels) What about newly developing cells? l Limit the search space to maximum expected storm movement l If mean absolute error is too large, assume that cell is new l Will take movement based on neighboring cells Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 18

4. Spatially interpolate motion vectors l l l Need motion estimate between regions l Spatially interpolate between regions l Weighted by distance from region (Cressman weights) l Weighted by size of region l Fill out spatial grid Can use background wind field to fill out domain l Constant weight for background wind field (from model) l Use scale=2 motion estimate as background field for scale=1 Repeat process to get motion vector for scale=2 l Use scale=1 motion estimate as background field for scale=0 l Repeat process to get motion vector for scale=1 Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 19

5. Kalman filter l Motion estimates are smoothed in time l Each pixel runs a Kalman filter (constant acceleration model) l Smoothes the motion estimates Courtesy: Yang et. al (2006) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 20

6. Use motion estimate to do forecast l l l Forward l Using motion estimate at a pixel, project the point to where it should be l Create a spatial Gaussian distribution of the point’s value at that location Interpolation l For fast moving storms, it is possible that there will be gaps in the output field l Interpolate between projected points Use different scales for different time periods, for example: l Use scale=0 forecasting less than 15 minutes l Use scale=1 forecasting 15 -45 minutes l Use scale=2 forecasting longer than 45 minutes Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 21

7. Trends l l What about trends? l Compute properties of current cluster l Min, max, mean, count, histogram, etc. l Project cluster backwards onto previous sets of images l Can use fields other than the field being tracked l Compute properties of projected cluster l Use to diagnose trends Not used operationally yet Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 22

Example: hurricane (Sep. 18, 2003) Image Eastward Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov Scale=1 s. ward 23

Satellite water vapor (Feb. 28, 2003) Image 30 -min forecast 60 -min forecast Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 24

Typhoon Nari (Taiwan, Sep. 16, 2001) Courtesy: Yang et. al (2006) l Composite reflectivity and CSI forecasts > 20 d. BZ l Large-scale (temporally and spatially) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 25

Tornado case (Apr. 20, 1995) Complete life-cycle of a storm: CSI at different scales and time periods Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 26

Tornado case (May 8, 2003) Courtesy: Yang et. al (2006) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 27

Comparison with other techniques (d. BZ) KTLX, May 3 1999 Bias MAE CSI Forecasting reflectivity through different techniques (30 min) 1. 2. 3. 4. Oct. 23, 2006 Persistence TREC (xcorr) Same wind-field for all storms Hierarchical K-Means + Kalman Valliappa. Lakshmanan@noaa. gov 28

Comparison with other techniques (VIL) KTLX, May 3 1999 Bias MAE CSI Forecasting VIL through different techniques (30 min) 1. 2. 3. 4. Oct. 23, 2006 Persistence TREC (xcorr) Same wind-field for all storms Hierarchical K-Means + Kalman Valliappa. Lakshmanan@noaa. gov 29

Forecast loop of VIL (May 3, 1999) Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 30

References l l l Technique described in this paper: l Lakshmanan, V. , R. Rabin, and V. De. Brunner, 2003: Multiscale storm identification and forecast. J. Atm. Res. , 67 -68, 367 -380 l http: //cimms. ou. edu/~lakshman/Papers/kmeans_motion. pdf Some of the results shown here are from: l Yang, H. , J. Zhang, C. Langston, S. Wang (2006): Synchronization of Multiple Radar Observations in 3 -D Radar Mosaic, 12 th Conf. on Aviation, Range and Aerospace Meteo. Atlanta, GA, P 1. 10 l http: //ams. confex. com/ams/pdfpapers/104386. pdf Software implementation l w 2 segmotion is one of the algorithms that is part of WDSS-II l Lakshmanan, V. , T. Smith, G. J. Stumpf, and K. Hondl, 2006 (In Press): The warning decision support system - integrated information (WDSS-II). Weather and Forecasting. l http: //www. wdssii. org/ Oct. 23, 2006 Valliappa. Lakshmanan@noaa. gov 31