Ocean Observatories Initiative OOI CI Overview and Status Matthew Arrott DMAC-ST Washington DC, Jan 18 -19, 2012 OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 1
Agenda • Review of the OOI CI Construction Objectives • Introduction of Release 2 Functional Objectives • Computational Infrastructure Objectives • External Observatory Integration • What will OOI-CI mean to me OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 2
OOI CI Construction Objectives OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 3
OOI Project Scope Building/deploying science infrastructure for understanding a changing water planet • Multi Dimensional Engineering Concern • Social • Organizational • Functional • Procedural • Technical Cyberinfrastructure fulfills the integrative mechanism across these dimensions OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 4
The Hubble Telescope for the Oceans OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 5
The Hubble Telescope for the Oceans OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 6
Core Science & Education Capabilities • Interactive Ocean Observing (R 3 - R 4) • Interactive Ocean Modeling & Data Assimilation (R 3 - R 4) • Discipline-Driven Semantic Organization of Data (R 3 - R 4) • Automated Data Product Generation (R 1 - R 3) • Interactive Instrument Network (R 1 - R 3) • Integrated Observatory Management (R 1 - R 2) • User-Driven Integration of Resource (R 1 - R 3) Observatory Activity Model 7 OOI CI Release 2 Kickoff May 2, 2011
OOI Integrated Observatory Product • The fully operational research observatory will meet the following • Goals: • • • Continuous observations at time scales of seconds to decades Spatial measurements from millimeters to kilometers Sustained operation during storms and other severe conditions Real-time or near-real-time data as appropriate Two-way transmission of data and remote instrument control Power delivery to sensors between the sea surface and the seafloor Standard plug-n-play sensor interface protocol Autonomous underwater vehicle dock for data download/battery recharge Access to deployment and maintenance vehicles that satisfy the needs of specific observatories Facilities for instrument maintenance and calibration A management system that makes data publicly available An effective education and outreach program OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 8
Extending the Traditional Data Access Model OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 9
Multi-Scale Research Specific Ocean Laboratories 10
Release Schedule Inception Phase Elaboration Phase Construction Phase Transition Phase 11 OOI Review Year 2 May 16 -20, 2011
Release Schedule by Subsystem Inception Phase Elaboration Phase Construction Phase Transition Phase 12 OOI Review Year 2 May 16 -20, 2011
CI Components Developed for R 1 • Python Capability Container • Java Capability Container Access Library • Web UI Platform • Exchange Messaging System • Distributed State Infrastructure • Resource Registry Framework • Science Data Persistence and Transport Format • Data Publish-Subscribe Framework • Event Notification Framework • Instrument Agent Framework • Elastic Processing Unit • Virtualized Cloud Management Tools 13 OOI Review Year 2 May 16 -20, 2011
Capabilities of the Managed Instrument Network • Operate Marine Observatories • Operate Platforms and Instruments • Manage Instrument Lifecycle • Test and Troubleshoot Instruments • Acquire Data and Generate Data Products • Search Data • Visualize Data • Manage the Integrated Observatory Network 14 OOI Review Year 2 May 16 -20, 2011
R 1 Integrated Observatory Network Web UI https: //confluence. oceanobservatories. org/display/CIPUB/OOI+Review+2011+Demo s 15 OOI Review Year 2 May 16 -20, 2011
Release 2 Functional Objectives OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 16
Observatory Operations & Administration OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 17
User Experience Incremental Development Plan • CI Release-1: Data Distribution Network • Provide framework for discovery and interaction based on data-intensive users’ needs • CI Release-2: Managed Instrument Network Science Experts • Provide support for operations and maintenance of Resource Providers marine networks based on RSN and CGSN needs General Science Users Scientific Collaboration • CI Release-3: On Demand Measurement Processing • Provide dynamic analysis and visualization tools based on science communities’ user needs • CI Release-4: Interactive Ocean Observatory • Provide mission control and collaboration support based on science communities’ needs 18 OOI Review Year 2 May 16 -20, 2011
R 2 Product Description: List of Use Cases UC. R 2. 01 Acquire External Data Source. UC. R 2. 02 Derive Data Product. UC. R 2. 03 Produce Real-Time QC Data. UC. R 2. 04 Browse to Get Data Product. UC. R 2. 05 Register and Connect Instrument. UC. R 2. 06 Command Instruments. UC. R 2. 07 Direct Instrument Access IIUC. R 2. 08 Manage Instrument Lifecycle. UC. R 2. 09 Activate Instrument Driver. UC. R 2. 10 Manage Marine Platform. UC. R 2. 11 Operate Marine Observatory. UC. R 2. 12 Deploy Agents On Remote Platform. UC. R 2. 13 Acquire Data From Instrument. UC. R 2. 14 Monitor an Instrument. UC. R 2. 15 Qualify Instrument Interface. UC. R 2. 16 Install Instrument Automatically UC. R 2. 30 Define Interaction. UC. R 2. 31 Define New Service. UC. R 2. 32 Conduct Negotiation. UC. R 2. 33 Enroll in an Org. UC. R 2. 34 Share an Org Resource. UC. R 2. 35 Share Affiliated Orgs' Resources. UC. R 2. 36 Create an Org. UC. R 2. 37 Control Service Interactions. UC. R 2. 38 Define Resource Life Cycle. UC. R 2. 39 Manage ION Users. UC. R 2. 40 Monitor ION Resources. UC. R 2. 41 Recover Failed Process. UC. R 2. 42 Define Resource Policy. UC. R 2. 43 Operate Message Brokers DM COI UC. R 2. 20 Annotate Resources. UC. R 2. 21 Transform Data in Workflow. UC. R 2. 22 Version CEI Resource. UC. R 2. 23 Ingest Dataset S&A Supplement. UC. R 2. 24 Search for Resource. UC. R 2. 25 Advanced Resource A&S UX Search. UC. R 2. 26 Navigate Resources and Metadata. UC. R 2. 27 Manage Replicated UC. R 2. 17 Define Visualization Archive. UC. R 2. 28 Manage Resource Method. UC. R 2. 18 Visualize Data Metadata. UC. R 2. 29 Integrate External Data Product. UC. R 2. 19 Produce Matlab Source Visualization 19 OOI Review Year 2 May 16 -20, 2011 UC. R 2. 53 View Modeler-Submitted Products. UC. R 2. 54 Access NEPTUNE CA Data Ops EOI UC. R 2. 55 Manage Help Ticket. UC. R 2. 56 Monitor ION Network. UC. R 2. 57 Configure Start Page UC. R 2. 44 Put Services Anywhere Easily. UC. R 2. 45 Replicate Activated Service. UC. R 2. 46 Operate Integrated System. UC. R 2. 47 Deploy Versioned User Process. UC. R 2. 48 Schedule User-Defined Process. UC. R 2. 49 Deploy Distributed Processes. UC. R 2. 50 Define Scaling Policy. UC. R 2. 51 Define Execution Engine. UC. R 2. 52 Manage ION Processes UC. R 2. 58 Display Arbitrary Resource. UC. R 2. 59 Generate New Screen
Managed Instrument Network OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 20
Sensor Set #1 Instrument Agents Class # Family Instrument Class Long Name ADCPS 16 Water Velocity_profile_600 m CTDGV 24 Conductivity, Temperature, Depth CTD_glider CTDMO 135 Conductivity, Temperature, Depth CTD_mooring CTDPF 30 Conductivity, Temperature, Depth CTD_profiler DOSTA 61 Dissolved Oxygen oxygen_dissolved_stable FDCHP 7 Direct Covariance Flux flux_direct_cov_HP FLORD 35 Fluorometer_two_wavelength METBK 18 Meteorology_bulk NUTNR 22 Nitrate nutrient_Nitrate OPTAA 35 Optical Properties attenuation_absorption_optical PCO 2 A 10 Partial Pressure CO 2 p. CO 2_air-sea PCO 2 W 25 Partial Pressure CO 2 p. CO 2_water PHSEN 33 Partial Pressure CO 2 p. H_stable PREST 3 Seafloor Pressure pressure_SF_tidal SPKIR 22 Optical Properties spectral_irradiance VEL 3 D 32 Water Velocity_point_3 D_turb VELPT 21 Water Velocity_point WAVSS 10 Surface Wave Spectra wave_spectra_surface ZPLSG 4 Zooplankton Sonar 21 OOI Review Year 2 May 16 -20, 2011 plankton_ZP_sonar_global
Sensor Set #2 Instrument Agents Class # BOTPT Family 3 Seafloor Pressure Instrument Class Long Name pressure_bottom_tilt CAMDS 12 Cameras x CAMHD 1 Cameras x camera_digital_video_HD HYDLF 5 Hydrophone_LF_passive MASSP 2 Mass Spectrometer mass_spectrometer OBSBB 3 Ocean Bottom Seismometer_BB_triaxial_accel OBSBK 2 Ocean Bottom Seismometer_BB_triaxial_keck OBSSP 8 Ocean Bottom Seismometer_shortperiod PPSDN 1 Seafloor Fluid and DNA_particulate RASFL 1 Seafloor Fluid and DNA watersample_chem_trace_H 2 S_p. H THSPH 1 Seafloor Properties Temp_H 2 S_p. H TMPSF 1 Seafloor Properties Temperature_seafloor TRHPH 1 Seafloor Properties Temp_resist 22 OOI Review Year 2 May 16 -20, 2011 camera_digital_still_strobe
Sensor Set #3 Instrument Agents Class # Family Instrument Class Long Name ADCPA 15 Water Velocity velocity_profile_mobile_asset ADCPT 17 Water Velocity_profile_300 m CTDAV 3 Conductivity, Temperature, Depth CTDBP 18 Conductivity, Temperature, Depth CTD_bottom_pumped DOFST 22 Dissolved Oxygen oxygen_dissolved_fastresp FDCLP FLORT 3 Direct Covariance Flux 42 Fluorometer CTD_AUV flux_direct_cov_LP Fluorometer_three_wavelength HPIES 2 Inverted Echo Sounder IES_pressure_velocity HYDBB 6 Hydrophone_BB_passive NUTR 4 3 Nutrients nutrient_four_channel OSMOI 2 Seafloor Properties watersample_chem_trace OTISF 1 Seafloor Properties flow_benthic PARAD 30 Optical Properties PRESF 9 Seafloor Pressure pressure_SF VADCP 2 Water Velocity_profile_50 m_turb ZPLSC 8 Zooplankton Sonar plankton_ZP_sonar_coastal 23 OOI Review Year 2 May 16 -20, 2011 PAR
CI Architecture for Instrument Integration 24 OOI Review Year 2 May 16 -20, 2011
Data Processing and Product Generation 25
OOI's Sensor Life Cycle OOI Final System Design Core Sensor List Core Sensor Specs Sensor Working Group Core Sensor RFPs Common Sensor Selections COL RSN COL CGSN Deploy at Sea RSN CI System Integration Test CI COL 26 RSN CGSN CI CI CGSN Procure Common Sensors OOI Review Year 2 May 16 -20, 2011 CI Test w/Instrumen t Test Kit Encode Algorithms Develop QC/Xform Algorithms RSN CI Integrate w/ Sensor Agent CI CGSN Develop Sensor Drivers
Instrument Development Kit • Scheduled for Release-2 • Includes: • Logical Test Facility Workbench (dry testing) • Marine Specific System Test Facility (wet testing) • Configurations for RSN and CGSN observatories • Access and Management Portals for Interactive access 27 OOI Review Year 2 May 16 -20, 2011
Computational Infrastructure Objectives OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 28
National Network Deployment (Year 3) 29 OOI Review Year 2 May 16 -20, 2011
Cyber. Po. Ps and Network Infrastructure • Acquisition Point Cyber. Po. Ps: • Portland, OR • Woods Hole, MA • Distribution Point Cyber. Po. Ps: • Seattle, WA • Chicago, IL • San Diego, CA • Mc. Lean, VA (optional) • Engineering Center: • San Diego, CA • Network Infrastructure • Dedicated 10 GE loop San Diego, Portland, Seattle, Chicago; branch to Mc. Lean (optional) • Dedicated 1 GE connection to Woods Hole 30 OOI Review Year 2 May 16 -20, 2011
San Diego Engineering Center • Purchased and deployed San Diego Cyber. Po. P and network equipment (see rack drawing) • Located in UCSD’s Atkinson Hall server room 1101 (secure and protected) • In use for • Release-1 production and QA • Management tools (Confluence, Jira, etc) 31 OOI Review Year 2 May 16 -20, 2011
San Diego Cyber. Po. P and Network Equipment 32 OOI Review Year 2 May 16 -20, 2011
Acquisition Point Cyber. Po. P Deployments (Y 3) • Sites • Portland • Woods Hole • Function • Data acquisition from marine observatories • Real-time data processing and OOI data product generation • Data preservation • High availability 33 OOI Review Year 2 May 16 -20, 2011
Cyber. Po. P Physical Layout (Year 3 Deployments) • Sites • Seattle • Chicago • Function • Content distribution (web servers, data servers, messaging) • Interconnects with major national and international network providers (layer 2 peering) • Links to compute clouds 34 OOI Review Year 2 May 16 -20, 2011
Network Peering Integration Seattle OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 Chicago 35
Multi-Tier Messaging Federation OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 36
Multi-Site Service Network Deployment 37
External Observatory Integration OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 38
IOOS Integration Model OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 39
External Observatory Integration Progress • Identification of datasets and data sources needed to support early adopter group (ESPRESSO/Rutgers, Hi. OOS/UHawaii) numerical model workflows • Classification of datasets by type and representation • Development of Dataset Agents for automatic data and metadata ingestion • Translation of data/metadata into the Common Science Data Format • Streaming of data packets and metadata update notifications 40 OOI Review Year 2 May 16 -20, 2011
External Observatory Integration • Exemplar: MARACOOS (Rutgers University) • Experimentally substitute part of the scientists’ data assimilation workflow preceding numerical model execution 41 OOI Review Year 2 May 16 -20, 2011
External Observatory Integration: Master Dataset List • Datasets • NAVY : NRLSSC (DAP) : 1 • NOAA : NDBC SOS (HTML) : 34 • NOAA : NDBC SDF (DAP) : 1 • NOAA : PFEG (DAP) : 1 • Rutgers : tashtego (DAP) : 2 • UCSD : HFRNET (DAP) : 1 • UH : SOEST (DAP) : 1 • USGS : Water. Service (HTML) : 26 42 OOI Review Year 2 May 16 -20, 2011
“What will OOI-CI mean to me” OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 43
Regional Data Provider & Manager • Effective mechanism to reliably at scale distribute your data • to your user community and data management/preservation facility • Independently publish and receive data • in a wide variety of syntactic formats and community vocabularies • Publish and receive as near-real-time data streams and as historical data sets • Effective mechanism to reliably at scale aggregate and disseminate data from your publishers and to your end user communities 44 OOI CI Release 2 Kickoff May 2, 2011
Participating Agency • Participate as providers, managers and consumers • Anticipated some requirements for publishing on the OOI as a separate Agency will require coordination with NSF • Requirements are being addressed as a part of the IOOS OOI Integration effort 45 OOI CI Release 2 Kickoff May 2, 2011
How does one “join” – approval / certification? • Open data policy network for consumers of data • Anticipate publishers required to register and certified • Data policies and rights management being established • Open data policies and content limitations to science and education data content • Publishing requires acceptance of some form of open use policy • Consuming requires some form of “as is”; “at your on risk”; “with attribution of source if republished” policies 46 OOI CI Release 2 Kickoff May 2, 2011
Relationship with IOOS, NEPTUNE and others observatories • External Observatory Integration subsystem within the OOI • Specifically focused on providing bi-directional integration interfaces to IOOS, NEPTUNE-Canada and WMO (aka GTS) • Integration continues on into the Operations and Maintenance phase of the OOI as a user/observatory “on boarding” capability • Provide the training and support to communities that wish to integrate with OOI 47 OOI CI Release 2 Kickoff May 2, 2011
OOI define standards for them to “plug in”? , e. g Sensor. ML • OOI provides a wide variety of syntactic formats and community vocabularies by which to interface with the OOI • OOI and the user community will be able to extend these interfaces • Capability facilitated by architectural choice of using controlled set of canonical data models into and out of which all data are transformed • New transformations are added, “plug ins”, to the network for a new format and/or vocabulary to one of the controlled data models 48 OOI CI Release 2 Kickoff May 2, 2011
Will a region be able to host an instance of the OOI “stack”? • The OOI “Integrated Observatory Network” is a federation architecture • Comprising participants from multiple domains of authority operating from their local domain of authority • The OOI “stack” exists at a couple of levels: • It is a messaging protocol • It is a component software implementation that implements the messaging protocol and process management containers (think modern web server) that publish and consume services • “Capability Containers”, can be and are written in multiple languages • achieve their interoperability through use of the common OOI messaging protocol • OOI message protocol and any of the OOI supplied Capability Containers will be provided under one of the standard open source licenses 49 OOI CI Release 2 Kickoff May 2, 2011
Thanks! OOI CI R 2 Life Cycle Objectives Review Aug 30 - Sep 1 2011 50
Скачать презентацию Ocean Observatories Initiative OOI CI Overview and Status
cd41b1729943ad6419f63bf195acb597.ppt