New Technologies for Storage and Display of Meteorological

New Technologies for Storage and Display of Meteorological Data Dale Paynter Operations Management Group May, 2002 NUMUG Conference 10 May 2002 OMG

Benefits of Technology? z Move to modern technology path that will be developed and supported z Make initial installation, maintenance & upgrades easier y Change is the name of the game x. External requirements & standards x. Infrastructure changes x. Organizational re-alignment y Meet rising expectations x. Simple systems interfaces x. Capitalize on staff knowledge & skills z Let Meteorologists manage, analyze and use the data 10 May 2002 OMG 2

New Technologies What are they? z Data Driven System Architecture z Relational Databases z Web Applications 10 May 2002 OMG 3

Data Driven Architecture What is it? Write code to be independent of specifics Make system configurable to define: y. Input stream y. Parameters for internal calculations y. Reports y. Interfaces to other systems 10 May 2002 OMG 4

Data Driven Architecture What are the benefits? z Respond to new requirements and standards z Reduce dependency on vendors & IT z Get out of program change queue z Reduce change orders & site visits z Give users control through configuring system 10 May 2002 OMG 5

What does this let you do easily? z. Add new sensors with no code changes z. Create new list reports z. Change corporate information z. Revise alarm / flag limits z. Add or decommission sampling locations z. Interface to different systems 10 May 2002 OMG 6

So how do we do this? Parameters, parameters. . . z Make data base ‘self-descriptive’ y. Store ‘what’ is being stored as well as ‘values’ y. Eliminate need to reorganize / reload data to accommodate changes z Make variables for all ‘constants’ y. Tower names y. Time zones y. Daylight saving time y. Measurement codes 10 May 2002 y. Report titles & headings y. Data to display / order y. Report field formats y. Conversion formulae OMG 7

Adding a new sensor z Traditional “Programming” Approach z Data Driven Architecture Approach y User configures or reprograms datalogger y Programmer adds new field to file formats. Then dumps existing data, reloads and verifies y Programmer changes data loading to process new input layout & update new field y Programmer changes updating to display / edit new field y Programmer changes reporting to display new field y System administrator installs final programs on all affected computers 10 May 2002 OMG y y User configures datalogger User defines sensor in database User tests and verifies results System administrator copies configurations to production system 8

Relational Databases What are they? z Data is stored in Tables made up of rows of fields (columns) z Data is referenced through name of the field z Subsets of data may be selected z Tables may be joined to retrieve related records 10 May 2002 OMG 9

Relational Databases Usage comparison example z Sequential file - 30’ wind speed is in columns 26 to 31 (format F 6. 2) z Typical relational database - 30’ wind speed is stored in a field named WS 30 z Relational database using data driven architecture - 30’ wind speed is stored in a field named Value where the field Meas. Code contains ‘WS’ and Height contains ‘ 30’ 10 May 2002 OMG 10

Relational Databases What are the general benefits? z Built-in security z Supports simultaneous accesses z Built-in data recovery techniques z Built-in data retrieval algorithms & optimization z Standard database interaction (SQL) z In-house expertise can be leveraged 10 May 2002 OMG 11

Relational Databases And. . . for meteorological data? z Concurrent loading, updating and reporting z Permits non-sequential loading for legacy data and data recovery z Handles data retention and archiving requirements z Ad hoc queries z Time-stamping data changes z Keep original values as well as edited values 10 May 2002 OMG 12

Web Applications In general. . . z User requests report and submits parameters through browser z Web application generates report on server and sends to browser z User views / prints report through the desktop browser Benefits? z Removes need to run / maintain specific report programs on desktop z Just ask your LAN Administrator… 10 May 2002 OMG 13

Web Server Configuration Meteorologist’s Workstation running Updater Meteorological Monitoring System Server Web Server MMS Database Sensors Zenos Intercept. TM & Loader Zeno®s 10 May 2002 OMG Reporter Interface Files Meteorologist’s & Users’ Workstations running Web Browsers Other Servers 14

Web Technology Internet or Intranet Server z. Data manipulation and report generation performed on server z. Only results sent to browser z. Automatic report refresh option z. Additional security levels possible 10 May 2002 OMG 15

How does it perform? Even running on a notebook: z. Current reports are instantaneous z. Historical reports take a few seconds z. Frequency Distribution produces a 1 year report in 12 seconds z. Data Capture Summary produces a 1 year report in 55 seconds 10 May 2002 OMG 16

Benefits of Technology z Move to modern technology path that will be developed and supported z Get out of the program change queue z Flexibility to respond to change z Let Meteorologists manage, analyze and use the data 10 May 2002 OMG 17