Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott

Ocean Observatories Initiative OOI Cyberinfrastructure Overview Matthew Arrott VMware Presentation March 5, 2010

Goals for this Presentation OOI CI LCO Review, Feb 2010 2

Overview • • The OOI Integrated Observatory OOI Network Architecture OOI Logical Architecture Conclusions OOI CI LCO Review, Feb 2010 3

OOI Program Overview OOI CI LCO Review, Feb 2010 4

OOI Integrated Observatory OOI CI LCO Review, Feb 2010 5

Integrated Observatory OOI CI LCO Review, Feb 2010 6

Science & Education Operations • Interactive Ocean Observing • Interactive Ocean Modeling & Data Assimilation • Automated Data Product Generation • Discipline-Driven Semantic Organization of Data • Interactive Instrument Network • Integrated Observatory Management • User-Driven Integration of Resource Observatory Activity Model OOI CI LCO Review, Feb 2010 7

Goal after Release 5 10

Network Architecture OOI CI LCO Review, Feb 2010 12

Network Architecture OOI CI LCO Review, Feb 2010 13

International Research Networks OOI CI LCO Review, Feb 2010 14

National Research Networks OOI CI LCO Review, Feb 2010 15

OOI National Network OOI CI LCO Review, Feb 2010 16

Network Deployment OOI CI LCO Review, Feb 2010 17

OOI Logical Architecture OOI CI LCO Review, Feb 2010 18

Release-1 OOI CI Functions. Interface Adapters Infrastructure Application • Instrument and platform interfaces • • • Instrument and platform control from the observatory Direct access to instruments and platforms Data acquisition from sensors and external observatories • • • Management of science data and other observatory information Data inventory and metadata annotation Data ingestion, i. e. canonicalization and metadata annotation Data discovery, cataloging and presentation Management Data distribution, replication, streaming Data persistence, assurance, retrieval • • Process and service execution Virtualization of computing Elastic scaling to demand High availability services Common Execution Infrastructure • • Service-orientation Reliable message based asynchronous communication Security: identity and policy management Multi facility system integration Common Operating Infrastructure Sensing & Acquisition OOI CI LCO Review, Feb 2010 19

The Capability Container Ingredients: • Capability: function, service, component • Plumbing: Messaging, security • Container: drop it all in as needed • Packaging: label it, tape it shut… done! OOI CI LCO Review, Feb 2010 20

Secure Reliable Messaging Capability (Service) Capability (Service) Capability Container Capability Container OOI CI LCO Review, Feb 2010 21

The “Exchange” • • Applications communicate through Exchange Spaces are namespaces of “communicators” Applications need to enroll in Exchange Spaces Governance applies within Exchange Spaces OOI CI LCO Review, Feb 2010 22

Exchange Space and Points • Exchange Points are the message routing and queuing resources of Exchange Spaces • Communicators play the role of Producer, Consumer, and Distributer (Broker) OOI CI LCO Review, Feb 2010 23

Exchange Space • Exchange Space is comprised of – Distributed Application Facility (DAF) – Distributed IPC Facility (DIF) OOI CI LCO Review, Feb 2010 24

Messaging Abstraction Send Message From: “name 2” To: “name 4” In: Exchg-Space 1 Action: “invoke-service” <Args><Data> Capability Container OOI CI LCO Review, Feb 2010 25

Common Message Format • Capability Container – Provides message handling through interceptors • After a service sends a message, before it receives a message • Message signing and validation • Policy enforcement • Governance tracking – Provides a common message format for all CI messages • Based in ACL FIPA message structure • Content, encoding, ontology OOI CI LCO Review, Feb 2010 26

Services and Resources as Agents Resource Agent Capability Container Resource Agent Proxy Agent Capability Container OOI CI LCO Review, Feb 2010 27

Scenario • An instrument, a physical resource, is represented by an agent to the system and its users • Users request control of the instrument • Capabilities are projected into another domain of authority by a proxy agent OOI CI LCO Review, Feb 2010 28

COI Infrastructure Services The Common Operating Infrastructure (COI) is the integration & communication environment for all the other subsystem services OOI CI LCO Review, Feb 2010 33

Integration Strategy • • • Capability Container Secure reliable messaging “Exchange” Service-oriented architecture Non-central governance COI Infrastructure Services OOI CI LCO Review, Feb 2010 34

CI Interfaces and Subsystems OOI CI LCO Review, Feb 2010 35

Preparing Services for Deployment Application Point Ingest Point Acquisition Point Access Portal Storage Point Instrument Point OOI CI LCO Review, Feb 2010 36

Deployment Scenario OOI CI LCO Review, Feb 2010 37

Deployment Plan and Strategy • Deployment of capability containers in multiple execution environments by leveraging virtualization – The CEI will make it work • Cyber. Po. P (Cyberinfrastructure Point of Presence): physical deployment of one or multiple capability containers – Different types OOI CI LCO Review, Feb 2010 38

Available, Elastic Execution OOI CI LCO Review, Feb 2010 39

CEI Service Interactions Registry Planner Provisioner Controller (“Executive”) Fault Monitoring and Compensation Resource (Taskable) OOI CI LCO Review, Feb 2010 2970 -00001 OV 2 CEI 40

Scenario: Planner and Controller • The pattern Planner. Controller-Resource can repeat itself at multiple (nested levels) 2970 -00018 OV 2 CEI OOI CI LCO Review, Feb 2010 41

CEI Architecture Overview VM VM (Deployable Unit) Application Software (Deployable Type) EPU Worker (Operational Unit) EPU HA Service (OOI Application) Availability and Scaling Strategy OOI CI LCO Review, Feb 2010 42

CEI Architecture Overview OOI CI LCO Review, Feb 2010 43

CEI Service Interaction Deployable Type Registry Planner Monitoring Information VM info Exchange Provisioner Proc Info Controller Iaa. S Adapter Other Info Iaa. S Context Broker OOI CI LCO Review, Feb 2010 44

Elastic Processing Unit • Elastic Processing Unit (EPU): High. Availability Services with policy based scaling to demand EPU Worker N 1 EPU Worker N 2 … EPU Worker NX [N, Deployable Unit] EPU Controller Register HA service Msg to N OOI CI LCO Review, Feb 2010 45

EPU: Prototype 1 • EPU design and prototype: – Policy-configurable planner/scheduler – Scaling policy: size of work queue – All components integrated with Magnet/AMQP OOI CI LCO Review, Feb 2010 46

Governance OOI CI LCO Review, Feb 2010 47

Motivating Governance • Administering collaborations – Based on framing normative relationships among peers – Abstracting away from low-level details • OOI, broadly: many stakeholders; many resources; longevity of decades • Exchange spaces, narrowly: abstractions for communicating; assembly of multiple topologies for messaging; analogous to traditional enterprise integration patterns OOI CI LCO Review, Feb 2010 48

Elements of a Service Engagement • Enactment: doing the domain work – what the end user cares most about • Administration: captured via contracts – Partnerships – Rules of encounter • Identity • Enforcement OOI CI LCO Review, Feb 2010 49

What is Governance? Broadly, administering service engagements • IT Governance: How IT resources are administered • SOA Governance: How services are created, deployed, removed, … • Currently, governance is manual – Low productivity – Poor scalability for fine-grained, real time governance decisions – Hidden, implicit considerations yield low confidence and poor maintainability OOI CI LCO Review, Feb 2010 50

Why Governance? • Stakeholders using resources to best serve individual and collective needs – Share resources in a controlled manner – Configure and reconfigure dynamically – Enable unanticipated uses for resources – Respect human organizational needs • In a nutshell, stakeholders administer themselves OOI CI LCO Review, Feb 2010 51

Separation of Concerns • Protocol: specifying the interactions among autonomous parties • Policy: specifying the decision making of each autonomous party as it participates in various protocols • Behavior: specifying the implementation that realizes the interactions OOI CI LCO Review, Feb 2010 52