The Use of AMET and Automated Scripts for

The Use of AMET and Automated Scripts for Model Evaluation Kristen Foley K. Wyat Appel Office of Research and Development National Exposure Research Laboratory, Atmospheric Modeling Division, Applied Modeling Research Branch

Sharing evaluation tools through EPA’s CMAQ Git. Hub Repository • As of October 2016, utility programs to process and prepare data for model evaluation are available on EPA’s Git. Hub repository for CMAQ: github. com/USEPA/CMAQ/tree/5. 2 Beta/POST Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 2

Sharing evaluation tools through EPA’s CMAQ Git. Hub Repository • Last CMAQ release included 9 utility programs, including updated “sitecmp” and “sitcmp_dailyo 3” programs for matching observed and model data. github. com/USEPA/CMAQ/tree/5. 2 Beta/POST Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 3

Sharing evaluation tools through EPA’s CMAQ Git. Hub Repository • Each post processing program in the repo includes the source code, sample run scripts and documentation. • As the CMAQ Team develops tools in-house to facilitate model evaluation, we will continue to add to this repository. • We are seeking feedback on the types of utilities that would be most beneficial to the larger model evaluation community. Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 4

Sharing evaluation tools through EPA’s CMAQ Git. Hub Repository • Current efforts include: • Automatically generating evaluation plots after a simulation completes with and without a dependency on AMET (shell scripts, FORTRAN, R) • Interactive spatial and time series plots for model evaluation (shell scripts, R) • 3 D model-obs pairing and evaluation plots for airborne and remote-sensed observations (shell scripts, R) Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory • Process analysis run script and standard evaluation plots (planned) 5

What is AMET? • Atmospheric Model Evaluation Tool • Two modules: Meteorology and Air Quality • Combination of several open source software packages – My. SQL – Fortran −R − Perl (will be removed in the next release) • AMET is specifically designed to compare observations against meteorological (e. g. MM 5, WRF) and air quality model (e. g. CMAQ, CAMx) predictions • Evaluation options include: summary statistics, scatter plots, spatial plots, time series plots, stacked bar plots, many other analyses Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 6

Advantages of the AMET System • Automated/interactive system • Data stored in relational database – Data from multiple simulations stored in a single location – Allows data queries based on many factors • Time period, geographic location, time of day, etc. • Pre-generated analysis scripts – Same analysis for multiple simulations – Common analyses between different groups • Open Source – Easy to create new scripts (if you know R) Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 7

Examples of AMET output Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 8

Public Availability of AMET • AMETv 1. 2 is currently available for download on the CMAS website – Both Met and AQ versions available – Script based version • Met and AQ versions can be installed together or individually • Includes users guide, tutorial data and example output plots Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 9

Future Improvements to AMET • AMETv 1. 3 scheduled for release in June 2017 – removes the need for PERL – streamlined installation – enhanced functionality, new analysis scripts, bug fixes, etc. • AMET Batch post-processing – single script that will create a set of pre-defined plots and other output files – eliminates the need to run separate scripts for each analysis – contains some basic level of customization • Possible next steps: – R and shell scripts available in Git. Hub repo – Web based version? Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 10

Possible next steps for AMET • Web based GUI for AMET – Easy selection of plotting options and sub-setting of sites/locations – Development will require resources outside of our team Office of Research and Development Atmospheric Modeling Division, National Exposure Research Laboratory 11