USGS Shake Cast Delivering Earthquake Shaking Data To

USGS Shake. Cast Delivering Earthquake Shaking Data To The People Who Need It Philip A. Naecker Chief Scientist Gatekeeper Systems pan@gatekeeper. com

Shake. Cast Features and Architecture u Project Origins u Project Team u Issues with Current Delivery of Shake. Maps u Project Goals and Shake. Cast System Goals u Shake. Cast Terminology u Shake. Cast Functions u u u Prototype V 1. 0 Futures u Shake. Cast u u u Gatekeeper Systems Features and Architecture Protocols and Transport Database Development Environment Shake. Cast Features and Architecture– June 2003 Slide 2

Project Origins u Success u of Shake. Map Project Shake. Maps are data rich u Recurrent difficulties utilizing Shake. Maps Consumers have had difficulty with network configuration, firewalls, and data processing u Each consumer wants a different configuration of feed, requiring “hand holding” and creating a maintenance morass u Missed opportunities for data utilization u Significant risk that information system fragility might impact successful application of Shake. Maps in a major event u Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 3

Shake. Cast Project Team u What eventually became the Shake. Cast Team has been in conversation about these issues since 1998 u USGS: Dave Wald and Bruce Worden u Peter German Consulting geologist, seismic data processing expert u Creator of the CUBE system u u Robert Nigbor USC Professor u Long involvement in practical applications of shaking data (e. g. , Intel) u Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 4

Software Development Expertise u Gatekeeper Systems In business since 1993 selling software for utilities u Specialize in high performance, high availability systems for large database, GIS, and Internet applications u Customers include large utilities and municipalities u • City of LA, Santa Barbara, Pacifi. Corp, Las Vegas Valley Water District, Georgia Power, Contra Costa County, etc. Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 5

Shake. Cast Software Development Team u Bruce Worden – Shake. Map Software Development u Phil Naecker u u Civil Engineer, 10 years as engineering consultant focused on water and wastewater utilities Built large high performance data management systems for business and engineering applications since 1971 u Steve u u u 45 years experience building software Founded firm that sold critical software systems to DOD and related agencies for over thirty years Internet and Open Systems Expert u Dave u u Caine Burke 25 years experience building software Exceptional competence at building complex systems Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 6

Shake. Cast Problem and Solution u Current Problems u Shake. Cast Solution Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 7

Current Delivery of Shake. Maps u Custom solution for each organization Unknown failure modes u Not built by experts u Not built using latest Internet technologies u u Difficult to implement and maintain Significant work for USGS and consumer u Not conducive to adding new data products or technologies u u Each organization has home-grown notification system (or none at all)

EQ Systems 2002 Public/ Private Networks Instrument Network Seismic Processing Archives Public Networks Private Networks Shake. Map Processing Akamai Servers Users FTP over Public Networks Private Networks Consumer Information System Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 9

Goals: Shake. Cast Project u Provide rapid and reliable delivery of information about earthquake shaking to the people who need it u u Initial audience focuses on utilities, large facility owners, municipalities, and news outlets “Shaking Information” is broadly defined u Satisfy some of the needs for post-event activities such as refinement of response plans and after-the-fact response assessment by accurately recording who knew what when u Deliver an open technology platform for earthquake information delivery that facility managers and the USGS can depend on and feel comfortable in building upon

EQ Systems 2003 Public/ Private Networks Seismic Processing Archives Private Networks Event Notification System Shake. Map Processing Instrument Network Distribution System Akamai Servers Public Networks Users Processing and Notification Private Networks Shake. Cast Consumer Information System Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 11

Goals: Data Delivery and Notification u Delivery and Notification Must Be Extremely Reliable Multiple data sources u Multiple data paths u Robust testing protocols u Set and Forget Design u u Fast Under All Load Conditions u Essentially no training required for proper installation and maintenance Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 12

Goals: Other System Attributes u Firewall Friendly u System Must Be Complete Functional “Out of the Box” in a few hours u Deliver data and most commonly-used tools u u Extensible and Flexible Able to integrate with more advanced tools u Customizable by organizations and individual users u u Traceable and Auditable System and usage audit logs, consistent timestamps u Objective and reproducible notification actions u Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 13

Shake. Cast Features and Functions u Phased rollout of features u Functions provide infrastructure, not enforce policy on how Shake. Maps are used u Reliability and robustness are designed in from the beginning Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 14

Shake. Cast Technology Plan u Open u source using widely used tools Apache, Perl, Oracle, Windows, Internet Explorer u Heavily documented code and databases u Based on commodity technologies: Internet and Web u Email and SMS pagers u Relational databases u u Security Gatekeeper Systems and reliability designed in from the start Shake. Cast Features and Architecture– June 2003 Slide 15

Shake. Cast Project Phases u Phase 1 - Prototype u Phase 2 – Reference System u Phase 2 – Open Source Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 16

Shake. Cast Phase 1 - Prototype u Funded by USGS u Purposes are proof of concept, demonstration u Implemented by Gatekeeper Systems and Bruce Worden u Due Summer 2003 u Cal. Trans will be first testbed organization u SBC/Pac. Bell anxious to be next testbed u Basic functionality, no easy installation or customization, single hardware/software platform Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 17

Shake. Cast Phase 2 – Reference System u Shopping for funding now u Fully installable (goal is one hour if the server already has a web server and database) u Easily configurable u Multiple hardware/software platforms u Looking for early adopters u Expect network of dozens to hundreds of systems and tens of thousands of end users Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 18

Shake. Cast Phase 3 – Open Source u Shared development between user organizations u Enhancements to core system u Participation by universities u Commercialization and Extension Integration with internal systems u Improved data for shaking estimates u Readily available sources for fragility estimates u u Hoping for thousands of systems and millions of end users Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 19

Shake. Cast Terminology u Server: every Shake. Cast machine is a server u Upstream/Downstream: direction of data flow, although complex network topologies are allowed u Parameters: measures of shaking generated by Shake. Map system u Products: data files, in various formats and multiple parameters, moved between Shake. Cast machines u Grid File: Shake. Map grids containing the raw parameter data u Notification: Detailed electronic message about a specific event, system activity, or shaking level to a specific user or group of users Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 20

Shake. Cast Software Features u Reliably and automatically receive and process shaking data from Shake. Map u Let organizations and users define locations of interest (facilities) and set shaking thresholds (green, yellow, red) in multiple shaking metrics (acceleration, instrumental intensity, etc. ) u Reliably deliver to end users electronic notification of facility damage estimates in a prioritized, customized, easy-to-use form u Make maps and reports from local servers available via the Web u Easily integrate with consumer’s other IT systems u Provide for end-to-end testing and upgrades Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 21

Prototype Features u Receive, store and forward Shake. Maps and associated metadata in a reliable manner u Unpack Shake. Map grids into a relational structure u Notify users of shaking and Shake. Cast activity Email and pager u System administrator, Shake. Cast developer, and end user events u u Produce detailed log files of Shake. Cast and user activity Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 22

Prototype Features - Transport u Receive from multiple upstream Shake. Cast and Shake. Map servers u Versioned products u HTTP push or HTTP polling/pull u Can transport not only shaking data but system metadata: i. e. product types, message types, etc. u Basic filtering for Shake. Map feed u u Bounding rectangle Peak grid values u Retry and basic error handling u Basic test suites Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 23

Prototype Features – Notification u On events: map generated, errors, recall/cancel, product delivery u On Shake. Map parameters: magnitude, acceleration, etc. u On location-specific shaking for any Shake. Map parameter u On exceedence of facility fragility for any Shake. Map parameter (green, yellow, red) u Message format driven by easily customized templates, includes direct Web links Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 24

Prototype Features - Database u Most configuration information, all Shake. Map data, and all user data is stored in the database u Support for both Access and Oracle u Site administrator can access database using standard SQL and other standard tools such as MS Access, Visual Basic, Perl, etc. Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 25

Additional V 1. 0 Features u Consolidated notification u Professional documentation u Complete installation procedure with upstream registration u User configuration web pages u Full support for both Unix and NT Gatekeeper Systems u Deliver Shake. Map web pages locally u Enhanced error handling u Enhanced test procedures u Call out from Shake. Cast to private scripts to invoke site-specific functions u Automated end-to-end testing Shake. Cast Features and Architecture– June 2003 Slide 26

Features Futures u Richer notification options Support for multiple related events u More intelligent prioritization of messages u More complicated notification logic u Support for positive response (confirmation) u Richer web links with active web pages to help users manage large lists of facilities u u More database platforms certified u Open source shared development environment u Upstream reporting of Shake. Cast usage, server health and status, and test results Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 27

Shake. Cast Implementation u Implementation Principles Use open and familiar tools and protocols u Follow path of least resistance for network and security managers u Open, readable, understandable source code u u HTTP and HTTPS transport u Relational database for all data storage u NT Service/Unix Daemon for event loop processing Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 28

Software Development Environment u NT and Unix (but could be anything) u Perl for CGI (but could be anything) u Standard SQL (and optional SQL-based tools) u Source code management in CVS (Web-based source management system designed for shared development) u No C, VB, Java or compiled languages required, providing short development cycles, transparency, portability Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 29

Shake. Cast System Data Flow Shake. Cast Server http ftp/NFS Shake. Map Server http Shake. Cast Server Web Pages Shake. Cast Data and XML http smtp http Notification Messages http Data and Invocation Shake. Cast Server GIS Systems Control Systems Alarm Systems Email System Paging System Shake. Map Server USGS Systems Private Systems Users Email/ Pager Messages

Protocols and Transport Architecture u HTTP CGI scripts for response to Shake. Cast requests u Shake. Cast metadata encoded in simple XML u HTTP GETs for file delivery u Can easily use HTTPS if needed u Authenticated server-server exchange: MD 5 secured passwords for authentication u Separate passwords for each server pair u u New CGI routines can be easily added u Use Apache HTTP server, but could use others Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 31

Database Architecture u Fully normalized data model u High cardinality transaction data can be purged if necessary u Shared data elements have shared primary keys u Locally-generated data elements have locallygenerated primary keys u Multiple database platforms possible MS Access and Oracle currently supported u Can easily be extended to use Oracle Spatial or other high-end database features u Gatekeeper Systems Shake. Cast Features and Architecture– June 2003 Slide 32

Questions and Answers ? Gatekeeper Systems ? ? Shake. Cast Features and Architecture– June 2003 ? Slide 33