The Atmospheric Model Evaluation Tool K Wyat Appel

The Atmospheric Model Evaluation Tool K. Wyat Appel and Robert C. Gilliam 7 th Annual CMAS Conference, Chapel Hill, NC October 9, 2008 Office of Research and Development National Exposure Research Laboratory, Atmospheric Modeling Division, Applied Modeling Research Branch October 8, 2008

Acknowledgements • EPA – Steve Howard – Rob Gilliam – Jenise Swall – Alice Gilliland – Sharon Phillips • UNC Institute for the Environment – Alexis Zubrow – Saravanan Arunachalam Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

What is AMET? • Atmospheric Model Evaluation Tool • Two modules – Meteorology (Rob Gilliam) – Air Quality (Wyat Appel) • Combination of several open source software packages – MYSQL –R – Perl • AMET specifically designed to compare observations against meteorological (e. g. MM 5, WRF) and air quality model (e. g. CMAQ, CAMx) predictions – Does not export all gridded data to database Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

AMET-MET Evaluation Flow Chart Analyses Observations • NOAA ESRL MADIS Surface-based Wind Profiler Precipitation Upper-air Observation-Model Synchronization Model Output • MM 5 (Net. CDF) • WRF (Net. CDF) Match obs. with model values in time and space Generate database records Connect to database and insert records Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Model Evaluation Database My. SQL server database that stores all model-observation pairs in tables for access by analysis programs. Model Performance Plot (scatter, range, histogram and table) Diurnal Statistics Time series Spatial Statistics Time-height statistics Text Upper-air statistics Other, User-developed tools The My. SQL database is a standard, widely used and easily connectable database that allows users to easily connect and extract data using other software (Excel, Matlab, Perl, SAS, etc. )

AMET-AQ Evaluation Flow Chart Observations STN, IMPROVE, CASTNet, NADP, AQS, SEARCH, MDN Model Output • CMAQ (IOAPI) • MCIP (IOAPI) Model Evaluation Database Observation-Model Synchronization My. SQL server database that stores all modelobservation pairs in tables for access by analysis programs. Match obs. with model values in time and space using site compare and compare airs programs Uses either web interface or existing PERL scripts to create required My. SQL tables. • Uses Combine program Analyses Model Performance Plots Diurnal Statistics Time series Spatial Statistics Box Plots Scatter Plots Generate database records (FORTRAN) Populate Database (PERL) Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory Bar Plots Other, User-developed tools “Soccer Goal” Plots The My. SQL database is a standard, widely used and easily connectable database that allows users to easily connect and extract data using other software (Excel, Matlab, Perl, SAS, etc. ) Bugle Plots

Advantages of the AMET System • Somewhat automated/interactive system • Data stored in relational database – Data from multiple simulations stored in a single location – Allows data queries based on many factors • Time period, geographic location, time of day, etc. • Pre-generated analysis scripts – Same analysis for multiple simulations – Common analyses between different groups • Open Source – Easy to create new scripts (if you know R) Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

Available AMET-MET Analyses • Model Performance Summary • • • – includes various plots (scatter, box, etc. ) along with various statistics Timeseries Spatial Plots – various statistical values (e. g. NMB, NME, etc. ) Bar Plots – error, bias, etc. Rawindsonde Wind Profiler Aircraft Profiler Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

Model Performance Summary Temperature Wind Direction

Time Series 2 -m Mixing Ratio 2 -m Temperature 2 -m Wind Speed 2 -m Wind Dir.

Spatial Statistics

Wind Profiler – Model Comparisons

Aircraft Profile Comparisons (Potential Temp) Mean Absolute Error Distribution by level Aircraft Mean WRF Mean

Available AMET-AQ Analyses • Scatter Plots • • • – model to observation – model to model (at observation points) Summary Statistics (as csv text file) Spatial Plots – various statistical values (e. g. NMB, NME, etc. ) – concentrations (predicted, observed, and difference) Box Plots Stacked Bar Plot Bugle Plot Soccer Goal Plot Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

Scatter plots • Model vs. Obs • Model vs. Model • Multiple Networks – select statistics • Single Network – additional statistics • Ozone Specific • Temporal Averaging – monthly, seasonal, annual

Spatial Plots • Statistics – NMB, NME, Correlation, etc. • Concentrations – model, observation, difference • Sub-regions

Time Series Plots

Box Plots Diurnal Box Plot Monthly Box Plot

Stacked Bar Plots

Other Plots “Bugle” Plot “Soccer Goal” Plot

Public Availability of AMET • AMET is available for download on the CMAS website • • – Both Met and AQ versions available – Script based version – Extensive users guide included Contains most of the functionality shown here Met and AQ versions can be installed together or individually Includes tutorial data and example output plots Bugzilla available for AMET Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory

Future Improvements to AMET • Java interface – Currently under development – Runs AMET locally and accesses remote database – Interactive tabs – User friendly – Portable (to a degree) • Additional analysis scripts – Developed internally – Developed externally (user community) • More query options Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory