Ocean Information Technology New approaches to ocean data

Ocean Information Technology New approaches to ocean data management Neville Smith BMRC, Australia COD, Brussells, 25 -27 Nov

COD, Brussells, 25 -27 Nov The Motivation … The Issues l l l l We have only 5 -10% of the needed telemetry capacity. Data Assembly - no broad agreement on how we ensure coherent, integrated data sets. Lack of agreement on standards, formats and practices. Poor modes of data transport, both routinely for operational requirements and for science. Poor or inappropriate uptake of modern information technology. No proper assessment of the future need. …

COD, Brussells, 25 -27 Nov Some Foundations l Some good examples of D&IM at work IODE, JCOMM OBIS (Co. ML), Argo DM, ODINAfrica, US IOOS Data plans … l but poorly integrated • “not invented here” syndrome many realizations of the same function/form Too much in-house technology, too little out-sourced functionality • Too little fusion / cross-sector adoption No / little engagement with scientific community • Innovation not within – …

COD, Brussells, 25 -27 Nov Rationale (The Why) • Approach IT as you would a Science Program l l l Effective telecommunications Common standards, practices and protocols (metadata management) Data and product service matched to the participants and users Innovative data inquiry, access and delivery Intra-operability and interoperability …

COD, Brussells, 25 -27 Nov The Components … the “How” 1. 2. 3. 4. 5. 6. 7. 8. Telemetry Metadata Management Data assembly, data set integrity, quality control Data circulation and transport Archives and archaeology Applications and user interfaces Capacity enhancement, training Governance, oversight, metrics

COD, Brussells, 25 -27 Nov 1. Telemetry and Communications … getting the data “in” (instructions out? ) a) Availability and capability l Bandwidth, 2 -way comms, connectivity, … b) Requirements for climate, open ocean c) Requirements for coastal regions d) Polar / remote regions Bandwidth … enabling

COD, Brussells, 25 -27 Nov 2. Metadata Management Semantic Metadata • Data about data … • Its environment • Its structure Data • Its meaning • Keywords l Syntactic Metadata IODE, JCOMM (CBS) and others have made considerable progress

COD, Brussells, 25 -27 Nov 2. Metadata … l Developing an agreed vocabulary – – Perhaps agreeing on a Markup Language: Marine. XML or … (e. g. , IODE/ICES Project) An essential element for intra-operability, interoperability and exchange Non-physical / unconventional: a challenge Complying with mandated requirements Complete, consistent, searchable metadata – Keywords, geospatial references – Adopt standardized, shared vocabularies – Developing catalogues – How to do highly effective searches in distributed systems? l How to automate, encourage, … l

COD, Brussells, 25 -27 Nov 3. Data assembly, quality control, … l More formality, accepted procedures, … – – From Level 0 (raw) through Level 3 (analysed) TOGA, WOCE, JGOFS experience Performance level monitoring

COD, Brussells, 25 -27 Nov 3. Datum and Data Set Integrity l Develop methodology for uniquely identifying original data and variants • Argo is being used as one test-bed • Original data given and always retains “tag” • Variations referenced against original l Accreditation – Eg, at level 2: a) no QC; b) auto; c) scientific – Institute A, A++ system, esp. for c) • • Recognize value adding Recognize scientific involvement Agreed peer-review system … ?

COD, Brussells, 25 -27 Nov 4. Data Circulation and Service l l Timeliness, efficiency Timeliness Architecture for data/product serving and exchange • • l Data packets and sets arrive with standard metadata and “tag” Systems like IDD are used to “push” data to routine users The sources may be real-time or high-quality delayed-mode Each ocean data distribution centre has instruction sets for distribution; event driven Also request-reply (pull) servers – US Data Communication initiative …

COD, Brussells, 25 -27 Nov US IOOS Model Circuits, data conveyers

COD, Brussells, 25 -27 Nov Together New Functionality l l l The XML standard (or an equivalent) should ensure all data are properly described / characterized Some of the needed metacode can be generated automatically The “tags” allow identification and removal of duplicates There is immediate identification of non-conforming data insertion No need to be a major centre to participate (provider or user) Some traditional data archive (backup) could be automated

COD, Brussells, 25 -27 Nov 5. Archives, Centres Reference other sessions l Centres might specialise in a type of service l – – l Routine versus ad hoc / itinerant Sophisticated versus non-specialist Strong link to User Interface theme Catalogues and Portals: Modern Library Irreplaceable observations, data products of lasting value and associated metadata preserved in perpetuity – Accessible using standards, keywords – Integrated into real-time and delayed mode data transport mechanism l Data archaeology

COD, Brussells, 25 -27 Nov 6. The User Interface l Must make data and information: – Available, usable, accessible, … l Scope – Assess “external” user requirements – “Wizards” to simplify interface – Assess “internal” user requirements – Robust interfaces; guaranteed responses l Several existing innovative client interfaces – Live Access Server and derivatives – GIS Systems; Web Map Server

COD, Brussells, 25 -27 Nov E. g. , GODAE LAS Product Server: UK http: //www. nerc-essc. ac. uk/las Daily 1/9 o FOAM N Atlantic data Menu for UK FOAM & POLCOMS data SST for 7 Jun 2002

COD, Brussells, 25 -27 Nov 7. Capacity, “education” … Some of the greatest challenges are in the area of community outreach and organizational behavior rather than in technology! [US IOOS DCS SC] l Challenges – Breaking down traditional barriers – Capacity enhancement: even, “fair” – Adoption and acceptance – Uneven access to technology, people l Reference other presentations l

COD, Brussells, 25 -27 Nov Conclusion l l l There is broad consensus on the need to act We have an outline of the strategy: why, how Enormous potential to impact science and applications; not discipline bound Architects and engineers are already busy Critical that we have broad participation Starting now – http: //ioc. unesco. org/oit

COD, Brussells, 25 -27 Nov Thank you MOI?