Water Information System for the Management and Storage

Water Information System for the Management and Storage of Time Series Data Presented by: Steve S. Di Lonardo & James C. Alair September 15 th, 2009 West Point, New York Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Outline § Challenges associated with time series data § WISKI software as a solution § Improvements to NYC DEP time series data quality and access? § Case studies § Other potential applications Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Challenges of time-series data § Government & environmental agencies produce large amounts of data • Requires the establishment of information systems to acquire and store data • Limited or no built-in QA/QC capabilities • Often results in data silos • Limited data sharing / access between groups Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Time series data § As part of DEP efforts to improve and facilitate the availability of time series data: • Pilot study using WISKI software • Database manager software • Designed by Kisters North America Inc. Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI database manager • Infrastructure to automatically IMPORT, PROCESS, COMPUTE, STORE, and REPORT all time series data • Unlimited sites, stations, parameters and data storage (only limited by hardware) • Based on MS-SQL, Oracle, etc…. • Scalable from a single user to hundreds of users Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI database manager § Data file importing (current and archival) • All major data loggers formats • Telemetry data acquisition systems • SCADA systems • XML, ZRXP, & Comma delimited files • User defined time-series data Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Who uses WISKI? Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Applications – Types of Data • • • Surface water (temperature) Ground water (salinity) Meteorological (precipitation) Water quality (O 2) Stream discharge (flood management) Any time-series data! Water Quality New York City Department of Environmental Protection Bureau of Water Supply

How/Where does WISKI fit at DEP? § § LIMS – lab/field data SAS – historical lab/field data SCADA – real-time data (operations) WISKI • Compatible with LIMS, SCADA, SAS, ARCGIS • Organization of high frequency data files into a manageable user-friendly format Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI software at DEP • Purchased by NYC DEP (2005) – Used primarily by EOH Hydrology - Kensico streams • Advancements 2007 - Present – Kensico Streams – USGS gauge stations – Weather stations – Keypoints – Reservoir Elevation/Capacity • Water Security Initiative (EPA Grant) – Integral part of keypoint data acquisition and storage Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI as a software solution • • Access via BWS intranet to all DEP employees Automated data imports (excel, html, asci) Reduce need for data requests Data available in a working format Water Quality New York City Department of Environmental Protection Bureau of Water Supply

NYC Watershed & Distribution Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI Data Management System Data Computations • Data Conversions – By Formula – Through Rating Tables • Creates Summary Data – Min, Max, Mean, Total – Daily, Monthly, & Yearly Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI Data Reports § Standard Reports • Year End Reports • Monthly Reports • Rating Curve Reports • Discharge Measurement Summary • User-defined Reports Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI Data Management System • • • Time and user stamped data QA/QC Original imported time-series data cannot be edited Altered data maintains history User-defined plausibility flags (eg. Turbidity) Restrict sensitive data to certain user groups Water Quality New York City Department of Environmental Protection Bureau of Water Supply

WISKI Data Management System Tabular presentation of the Flags Graphical presentation of the Flags Comments and Remarks Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Case Studies & Potential Applications at NYC DEP Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Case Study: Sewage Spill Warning & Prevention Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Water Budgets / Storm Monitoring Site Name All Flow (cfs) Base Flow (cfs) Storm flow (cfs) Base Flow % Storm Flow % MB-1 10, 284, 699 3, 109, 881 7, 465, 052 30. 2% 72. 6% E 9 49, 745, 086 14, 065, 627 35, 679, 459 28. 3% 71. 7% E 10 11, 986, 940 2, 595, 844 9, 272, 129 21. 7% 77. 4% E 11 12, 840, 648 4, 407, 686 8, 432, 962 34. 3% 65. 7% Sum 221, 367, 437 76, 834, 411 144, 752, 056 34. 7% 65. 4% Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Event Detection Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Operational Shutdown Monitoring • Reservoir elevation • Discharge and turbidity • Plausibility checks for out of range values Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Flood forecasting • Create a flood forecasting system using existing gage data • Establish a web-based warning network for towns Flood warning Flood watch Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Data Modeling • Nodes: represent catchments, junctions or recording sites • Links: define how the information flows in the model Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Data analysis - Trends Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Water Meter Information • Efforts have been made by NYC to automate water meter data acquisition • WISKI could interface with the data stream and auto-generate reports • Provide a historical database for data analysis Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Summary • WISKI Software represents a state of the art tool for dealing with any time series data • Does not require special infrastructure – can interface with any data source • Significant improvement in QA/QC & data access • Success requires collaboration Water Quality New York City Department of Environmental Protection Bureau of Water Supply

Acknowledgements § § Kisters North America Kerri Alderisio Lorraine Janus DEP Management Water Quality New York City Department of Environmental Protection Bureau of Water Supply