CUAHSI Hydrologic Information System Update David R Maidment

CUAHSI Hydrologic Information System Update David R. Maidment (PI) David G Tarboton Ilya Zaslavsky Michael Piasecki Jon Goodall With support from collaborators, postdocs and graduate students: Rick Hooper, Jon Duncan, David Valentine, Tom Whitenack, Jeff Horsburgh, Bora Beran, Tim Whiteaker, Ernest To, Cedric David http: //www. cuahsi. org/his. html

Definition The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

CUAHSI HIS Update • Current Status of the Project • HIS 1. 0 Components – Water. One. Flow web services – Observations data model • Digital Watershed

CUAHSI HIS Update • Current Status of the Project • HIS 1. 0 Components – Water. One. Flow web services – Observations data model • Digital Watershed

HIS Progress • Phase 1 grant, 2. 5 year grant completed • Phase 2 grant, 5 -yr, $4. 5 M to Maidment (Texas) • Co-PI’s – – Ilya Zaslavsky (San Diego Supercomputing Center) David Tarboton (Utah State) Michael Piasecki (Drexel) Jon Goodall (Duke) • WATERS Testbeds serve as beta-testers for software • First community-wide distribution in 12 – 18 months (dependant on establishing support services)

CUAHSI HIS Components (in development) Project co-PI in Phase 2 http: //www. cuahsi. org/his. html Collaborator in Phase I

HIS Team and its Cyberinfrastructure Partners Government: USGS, EPA, NCDC, USDA CUAHSI HIS Industry: ESRI, Kisters, Microsoft HIS Team: Texas, SDSC, Utah, Drexel, Duke Domain Sciences: Unidata, NCAR LTER, GEON Super Computer Centers: NCSA, TACC

WATERS Network Information System HIS Team WATERS Testbed

HIS, WATERS and the CUAHSI Community Government: USGS, EPA, NCDC, USDA CUAHSI HIS Industry: ESRI, Kisters, Open. MI WATERS Network Information System HIS Team Domain Sciences: Unidata, NCAR LTER, GEON WATERS Testbed Super computer Centers: NCSA, TACC

International Partners European Commission Water database design and model integration (Harmon. IT and Open. MI) Government: USGS, EPA, NCDC, USDA CUAHSI HIS Industry: ESRI, Kisters, Open. MI WATERS Network Information System HIS Team Domain Sciences: Unidata, NCAR LTER, GEON WATERS Testbed Super computer Centers: NCSA, TACC CSIRO Land Water Resources Observations Network (WRON)

CUAHSI HIS Update • Current Status of the Project • HIS 1. 0 Components – Water. One. Flow web services – Observations data model • Digital Watershed

Water Data Water quantity and quality Soil water Meteorology Remote sensing Rainfall & Snow Modeling

Water Data Web Sites

NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC # agency_cd site_no dv_dt dv_va dv_cd USGS 02087500 2003 -09 -01 1190 Time series of USGS 02087500 2003 -09 -02 649 USGS 02087500 2003 -09 -03 525 streamflow at a USGS 02087500 2003 -09 -04 486 gaging station USGS 02087500 2003 -09 -05 733 USGS 02087500 2003 -09 -06 585 USGS has committed USGS 02087500 2003 -09 -07 485 to supporting CUAHSI’s USGS 02087500 2003 -09 -08 463 Get. Values function USGS 02087500 2003 -09 -09 673 USGS 02087500 2003 -09 -10 517 USGS 02087500 2003 -09 -11 454

Observation Stations Map for the US Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System USGS National Water Information System NOAA Climate Reference Network

Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

CUAHSI Hydrologic Data Access System (being built using HIS Server in collaboration with ESRI) EPA USGS NCDC NASA NWS Observatory Data A common data window for accessing, viewing and downloading hydrologic information

HIS Server • Supports data discovery, delivery and publication – Data discovery – how do I find the data I want? • Map interface and observations catalogs • Metadata based Search – Data delivery – how do I acquire the data I want? • Use web services or retrieve from local database – Data Publication – how do I publish my observation data? • Use Observations Data Model

HIS Server and Analyst HIS Server Implemented at San Diego Supercomputer Center and at academic departments and research centers HIS Analyst Web Services Implemented by individual hydrologic scientists using their own analysis environments Flexible – any operating system, model, programming language or application Sustainable – industrial strength technology http: //www. cuahsi. org/his/webservices. html Details of HIS Analyst are here

Point Observations Information Model http: //www. cuahsi. org/his/webservices. html USGS Data Source Streamflow gages Network Neuse River near Clayton, NC Sites Discharge, stage (Daily or instantaneous) Variables Values 206 cfs, 13 August 2006 • • • {Value, Time, Qualifier} A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

Example: Matlab use of CUAHSI Web Services % create NWIS Class and an instance of the class create. Class. From. Wsdl('http: //water. sdsc. edu/wateroneflo w/NWIS/Daily. Values. asmx? WSDL'); WS = NWISDaily. Values; % Site Info for Site of Interest siteid='NWIS: 02087500'; str. Site=Get. Site. Info. Object(WS, siteid, ''); str. Site. site. Info. site. Name ans = NEUSE RIVER NEAR CLAYTON, NC lat=str. Site. site. Info. geo. Location. geog. Location. lat itude long=str. Site. site. Info. geo. Location. geog. Location. lo ngitude lat = 35. 6472222 long = -78. 4052778

Variable and variable. Time. Interval str. Site. series. Catalog(1). series(: ). variable ans = variable. Code: '00065' variable. Name: 'Gage height, feet' units: 'international foot' ans = variable. Code: '00060' variable. Name: 'Discharge, cubic feet per second' units: 'cubic feet per second' str. Site. series. Catalog(1). series(: ). variable. Time. Int erval ans = begin. Date. Time: '1927 -08 -01 T 00: 00: 00' end. Date. Time: '2006 -10 -16 T 00: 00: 00'

get. Variable. Info varcode='NWIS: 00060'; var. Info=Get. Variable. Info. Object(WS, varcode, '') var. Info = variables: [1 x 1 struct] var. Info. variables. variable ans = variable. Code: '00060' variable. Name: 'Discharge, cubic feet per second' units: 'cubic feet per second'

Get. Values % Get. Values to get the data siteid='NWIS: 02087500'; bdate='2002 -09 -30 T 00: 00'; edate='2006 -10 -16 T 00: 00'; variable='NWIS: 00060'; valuesxml=Get. Values(WS, siteid, variable, bdate, edate, '');

Parse XML and Analyze % Parse the XML into a Matlab object to work with valuesobj=xml_parseany(valuesxml); . . . plot(date, flowval); datetick;

Data Sources Storet Extract NASA Ameriflux NCDC Unidata NWIS NCAR Transform CUAHSI Web Services Excel Visual Basic C/C++ Arc. GIS Load Matlab Applications http: //www. cuahsi. org/his/ Fortran Access Java Some operational services

Objective • Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them What we are doing now …. . NWIS request return NAWQA request return request NAM-12 request return NARR Michael Piasecki Drexel University

Semantic Mediator What we would like to do …. . Get. Values NWIS Get. Values generic request Get. Values NAWQA Michael Piasecki Drexel University Get. Values NARR HODM

CUAHSI HIS Update • Current Status of the Project • HIS 1. 0 Components – Water. One. Flow web services – Observations data model • Digital Watershed

Hydrologic Science It is as important to represent hydrologic environments precisely with data as it is to represent hydrologic processes with equations Physical laws and principles (Mass, momentum, energy, chemistry) Hydrologic Process Science (Equations, simulation models, prediction) Hydrologic conditions (Fluxes, flows, concentrations) Hydrologic Information Science (Observations, data models, visualization Hydrologic environment (Dynamic earth)

Data Cube A simple data model Time, T “When” D “Where” Space, L Variables, V “What”

Continuous Space-Time Model – Net. CDF (Unidata) Time, T Coordinate dimensions {X} D Space, L Variables, V Variable dimensions {Y}

Discrete Space-Time Data Model Arc. Hydro Time, TSDate. Time TSValue Space, Feature. ID Variables, TSType. ID

Terrain Data Models Grid TIN Contour and flowline

CUAHSI Observations Data Model • A relational database at Streamflow the single observation level (atomic model) • Stores observation data Precipitation made at points & Climate • Metadata for unambiguous interpretation Water Quality • Traceable heritage from raw measurements to usable information Groundwater levels Soil moisture data Flux tower data

Hydrologic Observations Data Model What are the basic attributes to be associated with each single observation and how can these best be organized? Data Source and Network Sites Variables Values Metadata Controlled Vocabulary Tables e. g. mg/kg, cfs e. g. depth Streamflow Landuse, Vegetation Depth of snow pack e. g. Non-detect, Estimated, Windspeed, Precipitation A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time Data Discovery See http: //www. cuahsi. org/his/documentation. html Metadata provide information about the context of the observation. Data Delivery Ernest To Center for Research in Water Resources University of Texas at Austin

Independent of, but coupled to Geographic Representation Arc Hydro HODM Feature Hydrologic Observations Data Model Monitoring. Point 1 Site. ID Site. Code Site. Name Latitude Longitude … 1 OR Coupling. Table Site. ID (GUID) 1 Hydro. ID (Integer) Waterbody Hydro. Point Hydro. ID Hydro. Code FType Name Junction. ID * Complex. Edge. Feature 1 Hydro. ID Hydro. Code FType Name Area. Sq. Km Junction. ID * Edge. Type Flowline Shoreline Hydro. ID Hydro. Code Drain. ID Area. Sq. Km Junction. ID Next. Down. ID Simple. Junction. Feature Hydro. Edge Hydro. ID Hydro. Code Reach. Code Name Length. Km Length. Down Flow. Dir FType Edge. Type Enabled Watershed 1 Hydro. Network Hydro. Junction Hydro. ID Hydro. Code Next. Down. ID Length. Down Drain. Area FType Enabled Ancillary. Role 1 *

Variable attributes Cubic meters per second L 3/T m 3/s Variable. Name, e. g. discharge Variable. Code, e. g. 0060 Sample. Medium, e. g. water Valuetype, e. g. field observation, laboratory sample Is. Regular, e. g. Yes for regular or No for intermittent Time. Support (averaging interval for observation) Data. Type, e. g. Continuous, Instantaneous, Categorical General. Category, e. g. Climate, Water Quality No. Data. Value, e. g. -9999

Stage and Streamflow Example

Daily Average Discharge Example Daily Average Discharge Derived from 15 Minute Discharge Data

Water Chemistry from a profile in a lake

ODM and HIS in an Observatory Setting e. g. http: //www. bearriverinfo. org

HDAS Website Portal and Map Viewer Information input, display, query and output services Web services interface HTML -XML Water. One. Flow Web Services e. g. USGS, NCDC WSDL - SOAP 3 rd party data servers Uploads Do wnl oad s Preliminary data exploration and discovery. See what is available and perform exploratory analyses Data acce ss throu gh web servi ces Data storage through web services GIS Matlab IDL Observatory data servers CUAHSI HIS data servers Splus, R Excel Programming (Fortran, C, VB)

CUAHSI HIS Update • Current Status of the Project • HIS 1. 0 Components – Water. One. Flow web services – Observations data model • Digital Watershed

Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

Digital Watershed Hydrologic Observation Data Geospatial Data (GIS) (Relational database) Digital Watershed Weather and Climate Data Remote Sensing Data (Net. CDF) (EOS-HDF) A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

NHDPlus for Region 17 E

NHDPlus Reach Catchments ~ 3 km 2 Average reach length = 2 km 2. 3 million reaches for continental US About 1000 reach catchments in each 8 -digit HUC

Reach Attributes • Slope • Elevation • Mean annual flow – Corresponding velocity • Drainage area • % of upstream drainage area in different land uses • Stream order

http: //www. daymet. org/

• Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive • Software standards for model linking • Uses model core as an “engine” • http: //www. open. MI. org

Open. MI Conceptual Framework All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data Time, T D Space, L Variables, V VALUES An application of the data cube to integrate simulation models Jon Goodall, Duke University

HIS as Open. MI Components To calculate storage, the model needs inflow and outflow NWIS Streamflow water balance model Daymet Precipitation To calculate storage, the model needs precipitation Goal: Link the National HIS web services with a simple water balance model using Open. MI as the mediator Trigger: Calculate storage

Watershed Hydrovolumes Hydrovolume Geovolume is the portion of a hydrovolume that contains solid earth materials USGS Gaging stations

Stream channel Hydrovolumes Residence time distributions Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa)

Integration of surface water and groundwater data • Describe the relationship between surface water features ( e. g. streams and waterbodies) with groundwater features (aquifers, wells). • Enable the connection with the surface water data model Hydro network In the future go to 3 D. . . Aquifers

Water One. Flow • We need a “Water One. Flow” – a common window for water data and models Federal State Local Academic Flow Model Precip • Advancement of water science is critically dependent on integration of water information