Real-Time Algorithms G J Huffman NASA GSFC Science

Real-Time Algorithms G. J. Huffman NASA/GSFC & Science Systems and Applications, Inc. presented by P. A. Arkin Univ. of Maryland College Park/ESSIC Outline 1. What is the Goal of “Real-Time” Precip? 2. Available Data 3. Combination Algorithms 4. RT Issues 5. Final Remarks

1. What Is the Goal of “Real Time” Precip? We have a diverse, changing, uncoordinated set of input precip estimates, with various • periods of record • regions of coverage • sensor-specific strengths and limitations No single estimator gives uniform fine time and space coverage over large areas and long periods of time • at specific times and places a single sensor might be most insightful • but combination data sets usually the best choice Seek the most detailed record of “global” precip; must balance • “best” local, instantaneous answer • robust, reliable, automated system • speedy production Not developing a Climate Data Record (CDR; i. e. , final consistent data set) • compatibility with CDR-level estimates is highly desirable • so, focusing on RT here, but keeping CDR standards in mind

2. Available Data Globally, the principal data sources are satellite-based • passive microwave (PMW) - very good correlation to precip - emission channels not useful over land - no operational estimates over frozen surfaces - current technology limited to leo satellites example 3 -hr PMW collection for 03 Z on 03 Feb 2005; sensor types shown by shading the zero values HQ (mm/d) 03 Z 03 Feb 2005 Zeroes colored as: TMI (white), SSM/I (light grey), AMSR-E (medium grey), AMSU-B (dark grey)

2. Available Data (cont. ) • infrared (and other channels) on geosynchronous (geo) satellites - excellent sampling - sense clouds, not precip - confuse clouds and surface in cold environments Generic data problems include • under-determined physical problem - not enough data to solve problem • beam-filling errors - precip varies across satellite footprints • data availability due to satellite, transmission, archive, and data-policy issues • non-uniform sampling by uncoordinated satellite orbits

3. Combination Algorithms – The State of the Art Inter-calibration of PMW • GPROF-TMI used for the TMPA-RT Lagrangian time interpolation of PMW • Kalman smoother improves on first-generation deterministic morphing scheme (CMORPH, GSMa. P) PMW-calibrated geo-IR • extends PMW inter-calibration to IR estimate • calibration interval/region an open issue • schemes vary from simple colder-rains-more (NRL, PMWIR, VARR) to high-end statistical (Hydro-Estimator, PERSIANN) Adjustment to a monthly satellite/gauge combination to control bias • even a simple climatological adjustment is helpful (TMPA-RT) Multiple runs to accommodate late-arriving data • operational only in TMPA: RT and end-of-month post-RT

3. Combination Algorithms – The State of the Art Inter-calibration of PMW • GPROF-TMI used for the TMPA-RT Bias and RMSE for daily TMPA-RT vs. CPC gauge for CONUS before (red) and Lagrangian time interpolation of PMW after (black) calibration to • Kalman smoother improves on first-generation deterministic Version 6 scheme TMPA morphing (CMORPH, GSMa. P) • slight degradation in winter, large improvement in PMW-calibrated geo-IR summer. • extends PMW inter-calibration to IR estimate • calibration is monthly, using • calibration interval/region an open issue 10 yr of data • schemes vary from simple colder-rains-more (NRL, PMWIR, VARR) to high-end statistical (Hydro-Estimator, PERSIANN) Adjustment to a monthly satellite/gauge combination to control bias • even a simple climatological adjustment is helpful (TMPA-RT) Multiple runs to accommodate late-arriving data • operational only in TMPA: RT and end-of-month post-RT

3. Combination Algorithms – New Developments Short-interval gauge • Xie et al. have shown benefit with daily gauges • multi-day (~5 -10 days) might be needed for stability in data-sparse areas • latency issues might limit gauges in RT Multi-spectral geo estimates • Behrangi et al. have shown benefit, using a neural net • visible channel needed for substantial impact - good for 1/3 -1/2 the day • seems that only a few channels are needed to get most of the impact • current data systems not set to provide multi-channel geo data Cloud development using geo data • geo data clearly depict cloud evolution between PMW overpasses, but how to pick up the time series information? • Bellerby et al. have shown a conceptual cloud development model Adjustment for orographic enhancement not currently captured by PMW • Over land, PMW only sees ice precip, but orographic enhancement often liquid • Shige et al. and others are looking at moisture convergence, slope/aspect • best ancillary data not a settled issue

3. Combination Algorithms – New Developments (cont. ) Sounding-based cloud volume proxies • GPCP Monthly, Daily use TOVS/AIRS-based estimates • use cloud top, cloud fraction, moisture profile to estimate a cloud volume, then regress against gauge data • modest skill, but functions over all surface types • in development by Huffman et al. • sample GPCP 1 -Deg Daily using AIRS outside 40°N-S

3. Combination Algorithms – New Developments (cont. ) Model estimates • the next “obvious” improvement • model precip tends to - “win” in cold season/regions - “lose” in convective cases - handle the diurnal cycle badly - over-forecast the occurrence of precip • Sapiano, Arkin, Huffman, … looking at this • IPWG-WGNE joint validation of observation- and model-based precip should provide a basis for progress

4. RT Issues – Latency “True” RT is considered to have latency < 3 hr Actual latency depends on • sensor-to-host delivery - downlink scheme - network routing - administrative embargoes - at NASA/GSFC we can’t get closer than ~ 4 hr • algorithm speed • update interval for Web site A shorter cut-off time has better latency, but uses less data • on a “bad” day, delays can dramatically reduce the available data • different classes of users have different thresholds - “operations” (severe and flash flood warning, model initialization) need < 3 hr - daily data users (flood/landslide analysis, crop forecasting) need < 12 hr - drought analysis, crop forecasting interested in a 5 -10 -day latency for improved estimates (all satellite + some gauge)

4. RT Issues – Observation Interval Lack of coordination across the international constellation of precip satellites causes gaps and bunching in observation times The current sampling in the second half of the UTC day (same as in the first half): 12 geo-IR leo-MW (+ TMI) X 15 X X XX X 18 X X X X XX 21 X X X X 00 X X X X • the gap around 00/12 UTC is ~ 4 hr - Lagrangian time interpolators have to wait for the longest gap, plus the data latency - any shorter data cut-off (as in GSMa. P RT) forces extrapolation - interpolated PMW precip loses to even simple IR estimates when > 1 hr away from the nearest PMW overpass • the underlined conical-scanners (green) are not currently used due to longstanding calibration issues (F 15, F 16, F 17) and a new launch (F 18) • TRMM precesses, so the TMI episodically fills and uncovers the gaps

4. RT Issues – Data Dropouts Current PMW over land only sees ice hydrometeors • frozen or icy surface prevents retrieval • most northern land drops out HQ (mm/d) 03 Z 03 Feb 2005 in winter Zeroes colored as: TMI (white), SSM/I (light grey), AMSR-E (medium grey), AMSU-B (dark grey) • as well, PMW drop-outs formally eliminate the IR calibrator - RT schemes usually patch together some (lower-skill) scheme to continue using IR Alternative estimators in development include • “surface-blind” sounding channels (150+ GHz) • sounding-based cloud volume proxies

4. RT Issues – Error Estimates In general, error estimation for precip is primitive • non-negative, intermittent, multi-scale nature of statistics makes a general expression for random error very challenging - GPCP already provides a grid of estimated random error with each precip field, and users are mostly confused about how to use that - Hossain et al. estimate a complete expression might require 9 parameters - more work is clearly required • bias estimates only exist for regions with validation data - a scheme for estimating bias for GPCP was recently proposed by Adler et al. The vision for both is a parameterized equation driven by • some global or regionally specified error parameters • detailed grids for a few error parameters The methodology would be the same for RT and post-RT data

4. RT Issues – Design Goals What statistics are preserved in the calibrations? • users care about means, fractional coverage, extremes, spatial patterns • the final product should preserve these statistics - simple averaging or regression don’t • sensors with “low” fractional coverage present a problem - can we “invent” precip to match the coverage of the calibrator? What is the minimum resolution that satellite data should be expected to provide? • PMW footprints have sizes of 6 to 35 km • geo-IR footprints have size 4 -6 km at nadir • for scale consistency, PMW should be disaggregated to the geo-IR scale before being used to calibrate the geo-IR at full res. Users require an archive of reprocessed data when the RT algorithm changes • Current RT systems aren’t set up to reprocess

5. Final Remarks Several groups now have years of experience computing RT precip The state of the art is still rapidly improving • combined-satellite schemes critically depend on the quality of the input precip estimates • error estimates are a work in progress We desperately need every scrap of precip-relevant satellite data that we can get • “old” satellites provide important supplements to the (sparse) constellation of “new” satellites • updated radiometric calibration, applied to the entire archive of data is critical to the best progress in both RT and post-RT work Gauge data continue to be highly relevant as anchor points for the satellites User requirements are evolving as combination datasets evolve E-Mail: Web pages: george. j. huffman@nasa. gov http: //precip. gsfc. nasa. gov, http: //trmm. gsfc. nasa. gov