Processing tools — AAPP Nigel Atkinson Met Office

Processing tools - AAPP Nigel Atkinson (Met Office, UK) © Crown copyright Met Office EUMe. Train polar satellite event week 2012

Who am I? • I work for the Met Office in Exeter, UK • I’ve been in Met Office since 1982 and involved in satellite instruments (specifically AMSU-B) since early 1990 s • AAPP co-ordinator since 2003 • You may have met me at conferences such as ITSC or NOAA Direct Broadcast. • http: //www. metoffice. gov. uk/research/people/nigel-atkinson © Crown copyright Met Office

Who are you? AAPP users include: • National met. services • Universities • Research institutes • Military • Industry (e. g. direct broadcast station manufacturers) • Individuals • Others? yo e rp u r! te in o Us © Crown copyright Met Office EUMe. Train polar satellite week 2012

Contents • What is AAPP? • Examples of use with direct broadcast data (NOAA and Metop) • Interfaces with other DB packages • NPP processing • Visualisation • Conclusions and questions © Crown copyright Met Office EUMe. Train polar satellite week 2012

What is AAPP? • ATOVS and AVHRR Pre-processing Package • Software package for processing various types of polar orbiter data – imager and sounder • Maintained by the EUMETSAT Satellite Application Facility for Numerical Weather Prediction (NWP SAF) • Web site: http: //www. nwpsaf. org • Available free of charge (registration is required) • Current version v 7. 3 (10 th Oct 2012) © Crown copyright Met Office EUMe. Train polar satellite week 2012

© Crown copyright Met Office EUMe. Train polar satellite week 2012

Main purposes of AAPP • To prepare polar-orbiter data for use in NWP Øradiances from sounder instruments – microwave and I/R Ømajor contributor to forecast accuracy • To prepare data for retrieval packages Øsuch as IAPP (Univ. Wisconsin) for HIRS+AMSU • As a step to creating images ØAVHRR Øcloud mask • For creating climate products © Crown copyright Met Office EUMe. Train polar satellite week 2012

Data inputs • Raw data from satellite direct broadcast – complete level 1 processing ØNOAA-15 -19, Metop-A/B, (FY-1 D) • Outputs from other DB packages Øe. g. Sensor Data Records from NPP (from University of Wisconsin’s CSPP) • Level 1 B data from NOAA or EUMETSAT Ønear-real-time or archived (e. g. NOAA CLASS) Level 1 B = raw counts (or radiances) on original instrument grid, with geolocation © Crown copyright Met Office EUMe. Train polar satellite week 2012

Requirements To run AAPP, you need • Linux or unix system, e. g. Øe. g. Linux PC, Sun Solaris, IBM AIX • A suitable Fortran compiler s! Not w indo W Øe. g. gfortran, g 95, ifort, pgf 90, xlf 90, …. ØMost parts of AAPP will also work with a fortran 77 compiler, e. g. g 77 • C compiler • Korn shell, perl, make, etc. • For IASI level 1 processing you also need C++, multithreading support and at least 2 Gb memory • External libraries are used for some parts of the processing (e. g. hdf 5, BUFR, GRIB) – see Installation Guide for details © Crown copyright Met Office EUMe. Train polar satellite week 2012

Example: NOAA-19 direct readout (one of the test cases supplied with AAPP) • Inputs Ø HRPT file: hrpt_noaa 19_20110906_0206_13280. hrp (113 Mb for 14 minute pass) Ø Orbital elements (spacecraft position information): tle_20110906_0255. txt § these can be obtained by the user from celestrak. com or www. space-track. org. § normally save them in AAPP directory AAPP/orbelems/tle_tb/2011 -09 • AAPP command AAPP_RUN_NOAA –i “AMSU-A MHS HIRS AVHRR” –g HIRS hrpt_noaa 19_20110906_0206_13280. hrp • Outputs amsual 1 b_noaa 19_20110906_0206_13280. l 1 b amsual 1 c_noaa 19_20110906_0206_13280. l 1 c hirsl 1 b_noaa 19_20110906_0206_13280. l 1 b hirsl 1 c_noaa 19_20110906_0206_13280. l 1 c hirsl 1 d_noaa 19_20110906_0206_13280. l 1 d hrpt_noaa 19_20110906_0206_13280. l 1 b mhsl 1 b_noaa 19_20110906_0206_13280. l 1 b mhsl 1 c_noaa 19_20110906_0206_13280. l 1 c © Crown copyright Met Office AMSU-A HIRS AVHRR MHS Takes a few seconds on a modern linux system Binary files, with header record, and 1 data record per scan EUMe. Train polar satellite week 2012

Outputs rendered as images MHS AMSU Used IDL for these images Many possibilities AVHRR HIRS © Crown copyright Met Office EUMe. Train polar satellite week 2012

Example 2: Metop-A direct readout • Inputs Ø Level 0 files: AMSA_HRP_00_M 02_20100128104249 Z_20100128105130 Z_N_O_20100128104314 Z AVHR_HRP_00_M 02_20100128104259 Z_20100128105149 Z_N_O_20100128104305 Z HIRS_HRP_00_M 02_20100128104253 Z_20100128105138 Z_N_O_20100128104316 Z HKTM_HRP_00_M 02_20100128104256 Z_20100128105143 Z_N_O_20100128104310 Z IASI_HRP_00_M 02_20100128104259 Z_20100128105148 Z_N_O_20100128104303 Z MHSx_HRP_00_M 02_20100128104252 Z_20100128105138 Z_N_O_20100128104310 Z Ø Orbital elements: as before, or get TLEs from HKTM Ø For IASI processing, need auxiliary files (available on AAPP ftp site) • AAPP command export PATH_OPS=. . . ; export DIR_IASI_CONFIG=. . . AAPP_RUN_METOP –i “AMSU-A MHS HIRS AVHRR” –g HIRS AAPP_RUN_METOP –i “AMSU-A MHS AVHRR IASI” –g IASI • Outputs amsual 1 b_M 02_20100128_1042_17002. l 1 b amsual 1 c_M 02_20100128_1042_17002. l 1 c hirsl 1 b_M 02_20100128_1042_17002. l 1 b hirsl 1 c_M 02_20100128_1042_17002. l 1 c hirsl 1 d_M 02_20100128_1042_17002. l 1 d hrpt_M 02_20100128_1042_17002. l 1 b iasil 1 c_M 02_20100128_1042_17002. l 1 c iasil 1 d_M 02_20100128_1042_17002. l 1 d mhsl 1 b_M 02_20100128_1042_17002. l 1 b mhsl 1 c_M 02_20100128_1042_17002. l 1 c © Crown copyright Met Office Takes a few minutes for IASI AMSU-A HIRS AVHRR IASI MHS IASI 1 d: - mapped AMSU - channel selection - PCs if requested EUMe. Train polar satellite week 2012

IASI level 1 c © Crown copyright Met Office EUMe. Train polar satellite week 2012

IASI level 1 processor • The IASI level 1 processor is called OPS-LRS: Operational Software – Local Reception Station • Optional addition to the core AAPP • Based on CNES/Thales code used in EUMETSAT ground segment • Adapted by NWP SAF for portability and to make it easier to install/run Øthough not very user-friendly if something goes wrong! • See OPS-LRS User Manual (on AAPP web page) With the recent launch of Metop-B, we expect interest in IASI from HRPT stations that are unable to receive Metop -A (e. g. South America) © Crown copyright Met Office EUMe. Train polar satellite week 2012

AVHRR cloud mask • This is another optional part of AAPP • Cloud mask “MAIA 3” developed by Météo-France (Lydie Lavanant), introduced Feb 2010. • It makes use of NWP forecast fields (GRIB), or climatology if these are not available. • Command: MAIA 3_RUN hrpt_noaa 19_20110906_0206_13280. l 1 b • Outputs avh_noaa 19_20110906_0206_13280. l 1 c - radiances avh_noaa 19_20110906_0206_13280. l 1 d - radiances + cloud mask and optionally plots like this one … © Crown copyright Met Office EUMe. Train polar satellite week 2012

Cloud mask product © Crown copyright Met Office EUMe. Train polar satellite week 2012

What about DB from other polar satellites? Satellite Software Available from Interface with AAPP? Aqua/Terra MODISL 1 DB (Sea. DAS) http: //seadas. gsfc. nasa. gov/ No requirement IMAPP http: //cimss. ssec. wisc. edu/imapp/ (There is an IMAPP tool to generate AMSU-A BUFR files) RT-STPS (raw → RDR) and CSPP (RDR → SDR) http: //cimss. ssec. wisc. edu/cspp/ IPOPP (NASA Direct Readout Lab) http: //directreadout. sci. gsfc. nasa. gov Yes. AAPP can ingest the SDR files (hdf 5 format) FY-1 D (“Feng Yun”) AAPP NWP SAF (Satellite no longer working) FY-3 A and FY-3 B FY 3 L 0 pp and FY 3 L 1 pp http: //www. nsmc. cma. gov. cn/New. Site /NSMC_EN Yes. AAPP can ingest the SDR files (hdf 5 format) MODIS level 2 AIRS level 1 & 2 AMSR-E level 1 & 2 NPP Training → FAQs DMSP (U. S. military) No - - Meteor-3 M (Russia) No - - © Crown copyright Met Office EUMe. Train polar satellite week 2012

Recent launches Metop-B: 17 th Sept 2012 Suomi NPP: 28 th Oct 2011 © Crown copyright Met Office EUMe. Train polar satellite week 2012

Metop-B L 0 data from Judd Taylor, Orbital Systems • AMSU, MHS, HIRS, AVHRR are working (IASI later) • See NWP SAF Forum “AAPP Announcemnts” for code patches: www. nwpsaf. eu/forum © Crown copyright Met Office EUMe. Train polar satellite week 2012

© Crown copyright Met Office EUMe. Train polar satellite week 2012

AAPP support for Suomi NPP from AAPP v 7. 1 (Feb 2012) AAPP ingests the SDR files – hdf 5 or BUFR. Stores in “AAPP-format level 1 c” ATMS Cr. IS • Spatial filtering (noise reduction and channel matching) • Channel selection • Spatial thinning • BUFR encoding • Spatial thinning • Map to Cr. IS • BUFR encoding VIIRS • Cloud mask coming soon © Crown copyright Met Office EUMe. Train polar satellite week 2012

ATMS spatial filtering Two issues for NWP: 1. For many channels, raw NEΔT is larger than model background error (and larger than AMSU NEΔT) 2. Difficult to use channels 1&2 due to beamwidth mismatch with channels 3 -15 (5. 2° vs 2. 2°) These issues are handled by AAPP: • Broaden ch 3 -15 beam width to that of AMSU-A • Narrow ch 1 -2 (as far as possible without increasing noise) see AAPP document on ATMS processing at http: //www. nwpsaf. org © Crown copyright Met Office EUMe. Train polar satellite week 2012

Channel 3 -2 difference images Raw SDR Evidence of improved contrast and reduced artefacts Technique used operationally at Met Office and NCEP Modified © Crown copyright Met Office AMSU EUMe. Train polar satellite week 2012

Cr. IS • Assimilation trials underway at Met office • Noise looks good and biases are low © Crown copyright Met Office EUMe. Train polar satellite week 2012

VIIRS cloud mask • VIIRS cloud classification (MAIA 4) to be added to AAPP From Lydie Lavanant © Crown copyright Met Office EUMe. Train polar satellite week 2012

Use of AAPP in regional data distribution See WMO RARS page RARS = Regional ATOVS Retransmission Service • Each RARS node is a direct broadcast reception station • AAPP is run at each RARS node – products in BUFR • ATOVS data available within 30 minutes (normally via GTS) • IASI and ATMS/Cr. IS coming soon © Crown copyright Met Office EUMe. Train polar satellite week 2012

EUMETSAT’s initial EARSNPP network Svalbard ATMS – all channels & spots, BUFR Cr. IS – 399 channels, BUFR VIIRS – M-band, HDF 5 Kangerlussuaq Moscow Lannion Athens Maspalomas © Crown copyright Met Office EUMe. Train polar satellite week 2012

Visualisation • Visualisation tools are not officially part of AAPP Øbecause not needed for NWP • But visualisation of the microwave and I/R data can be very useful for Ønowcasting, verification, diagnostics, meteorological analysis, etc. • Many possibilities, some free, some commercial, e. g. ØIDL, Python, Matlab, Mc. IDAS-V, GMT, IDV, Vis. AD, Hydra, Metview, Visual Weather, ERDAS IMAGINE, etc. ØAny others? • Listener preferences? • Met Office has some IDL tools that we can supply on request (best effort basis) © Crown copyright Met Office EUMe. Train polar satellite week 2012

MHS imagery of Hurricane Isaac 28 Aug 2012, 03 Z (Metop-A) 89 GHz 157 GHz Strong ice scattering signal 190 GHz AVHRR chan 4 © Crown copyright Met Office EUMe. Train polar satellite week 2012

Scattering index (AAPP) + geo I/R These images are used by Met Office forecasters Local reception, EARS and global © Crown copyright Met Office EUMe. Train polar satellite week 2012

Thank you for listening! Visit http: //www. nwpsaf. org for more information Questions? © Crown copyright Met Office