Australian Marine data AODCJF and Blue Net Kate

Australian Marine data, AODCJF and Blue. Net Kate Roberts, Director Blue. Net Project AUKEGGS 3 rd Workshop - 29 November 2006

Blue. Net project • Funding to kick-start Blue. Net from DEST • Blue. Net aims to build a data-sharing culture within Universities. – Entails creation of the necessary support infrastructures (including guidelines, software, training, etc) for researchers in the University sector to participate in data-sharing and more effective data discovery and data re-use, into the future • Blue. Net partners (AODCJF, Blue. Net Project Office, APAC, APSR) are building a virtual data repository for delivery and longterm curation of Australian marine data – professionally-managed – state-of-the-art/ best practice – sustainable AUKEGGS 3 rd Workshop - 29 November 2006

Blue. Net background • Integrates marine researchers at Australian Universities into the AODCJF data sharing and discovery network • Management of Blue. Net is from the Project Office at the University of Tasmania • Staff are employed at member universities and AODCJF agencies to facilitate the flow of data AUKEGGS 3 rd Workshop - 29 November 2006

Blue. Net collaboration • Building a long-term Blue. Net involves development and/or promotion of: – – – Common legal agreements Guidelines Standards Protocols Sustainable practices • Blue. Net engaged in developing agreements, addressing IP/ licencing needs, addressing authentication/access issues, developing training and support; • Software development (MEST, Portal) AUKEGGS 3 rd Workshop - 29 November 2006

Blue. Net partners • AODCJF agencies • APAC/TPAC (Tasmania Partnership for Advanced Computing) • University Partners – – – – Australian National University James Cook University of Melbourne University of Sydney University of Queensland University of Western Australia University of Tasmania • Other universities will join as the project develops AUKEGGS 3 rd Workshop - 29 November 2006

Metadata activities • Metadata is a critical component for AODCJF to provide the mechanism for data discovery • AODCJF has developed a “marine community of practice profile” of ISO 19115 international standard; • Implementing a marine catalogue service as part of the Oceans Portal • Developing controlled vocabularies to describe data AUKEGGS 3 rd Workshop - 29 November 2006

Marine Community Profile Marine Profile • Includes all ISO 19115 core metadata components • Extends 19115 to include several new elements, and customised codelists to meet the need of the marine community • Is compliant with new ANZLIC profile AUKEGGS 3 rd Workshop - 29 November 2006

Metadata tool • AODCJF identified the need for a common Metadata Entry and Search Tool (MEST) • ISO 19115 compliant; • Ability to implement the Marine Profile, and other related profiles • To be adopted by most of the AODCJF organisations: replace existing non-ISO systems; – Make available to other marine data providers. AUKEGGS 3 rd Workshop - 29 November 2006

Metadata tool requirements • • • Web-accessible; Open source software; Allow (ISO 19115/ Marine Profile) metadata creation, validation, storage, edit, retrieval and display Importing /Exporting – in batch mode, or singly; Conversion between profiles (support use of conversion programs – internally, or by linking to external services); Conversion between vocabularies; Ease of updating vocabularies; Customised record-numbering/tagging; Allow easy update processes (update/overrwrite, replace/ remove earlier one, etc) Allow local badging, and other customisation, including additiion of fields needed for core business processes (eg financial details); AUKEGGS 3 rd Workshop - 29 November 2006

Metadata tool requirements • Facilitate complex boolean searching, and cross-field searching; • Federated searching – using range of protocols: HTTP, z 39. 50/SRW, OAI – Connection of individual MEST instances to other MESTs, and the ASDD • Support harvesting of metadata by external agents (e. g. into the ASDD and into the Marine Catalogue -for access by Ocean Portal) • Support Users-Levels, and management of varying permissions; (edit rights, view rights, delete rights, etc) • Connection to external authentication systems; • Freely available for distribution AUKEGGS 3 rd Workshop - 29 November 2006

Metadata tool development MEST development approach : staged process: Geo. Network software selected [for Stage 1]; Developer selected; Stage 1 Simple / immediate need • Import Marine Profile • Assess/ Report on robustness and suitability of software for Stage 2 • Gap Analysis; Stage 2 Address longterm community needs • Cross-searching, Harvesting, Authentication using remote authorities; • Complex boolean searching, and cross-field searching; etc AUKEGGS 3 rd Workshop - 29 November 2006

Metadata tool development Challenges • Software has strong XML emphasis…skills deficit • Documentation poor; • Error messages unclear; • Off-the-shelf version – not ISO 19115 compliant (contrary to PR); – Doesn’t allow use of schematron, in validation; • Newer version almost finalised; • MEST Development Reference Group; – Design choices / varying needs (eg store as XML blob, and/or as fields in relational DB) – Expertise on metadata use, or system design AUKEGGS 3 rd Workshop - 29 November 2006

International activities • The IOC/IODE community will implement a marine profile of ISO 19115 • The AODCJF Marine Community Profile is being considered • Will replace the current DIF format • MEST could be also be used by IOC • Recommendation to be made in March 2007 AUKEGGS 3 rd Workshop - 29 November 2006

International activities • Marine Metadata Interoperability (MMI) Sensor Metadata Interoperability workshop (October 06) • Goal: to establish consensus on a content standard to describe sensors in ocean observing systems. • AODCJF Marine Community Profile was one of five standards evaluated • One option: use the Marine Profile to describe data collected by sensors using Sensor. ML AUKEGGS 3 rd Workshop - 29 November 2006

Oceans Portal • Development and testing complete • To be deployed Dec. 2006 • Portal installed at DEH, Canberra • Marine Catalogue installed at CMAR, Hobart • Will draw on resources, expertise, support from AODCJF agencies, as well as from Blue. Net and other partners; AUKEGGS 3 rd Workshop - 29 November 2006

Oceans Portal Design • Metadata driven – data or information service must have a metadata record; the integrity of the Portal is based on the quality of the metadata • Discovery of any data or service – any data or information service that has its metadata record published will be discoverable by the user • Multiple methods of access – Access will be through the Portal or through other client applications that adhere to the relevant standards • Open standards – the Oceans Portal Project conforms to open standards • The Oceans Portal uses the publish-bind-find model AUKEGGS 3 rd Workshop - 29 November 2006

IMOS • Integrated Marine Observing System • Five year project starting 2007 • One of nine priority capability areas for National Collaborative Research Infrastructure Strategy (NCRIS) • IMOS Nodes/Facilities across Australia AUKEGGS 3 rd Workshop - 29 November 2006

IMOS “Investment” areas 1. 2. 3. 4. 5. Argo Australia Enhancement of Measurements on Ships of Opportunity Southern Ocean Automated Time Series Observations Australian National Facility for Ocean Gliders Australian National Autonomous Underwater Vehicle (AUV) Facility 6. Australian National Mooring Network 7. Australian Coastal Ocean Radar Network 8. Australian Acoustic Tagging and Monitoring System 9. Facility for Automated Intelligent Monitoring of Marine Systems 10. e. Marine Information Infrastructure 11. Enhancing Access to Australian Ocean Remote Sensing Data AUKEGGS 3 rd Workshop - 29 November 2006

AUKEGGS 3 rd Workshop - 29 November 2006

e. Marine Information Infrastructure e. MII aims to: • Provide a single integrative framework for data and information management, discovery and access for incoming IMOS data • Host, manage and archive the data from the IMOS nodes • Provide standards, protocols and systems to integrate the data and related information …. with a focus on e. Research needs. Therefore, international collaboration/ cooperation in the DM/IM space is even more important. e. MII collaborating institutions: – AODCJF – Blue. Net – TPAC AUKEGGS 3 rd Workshop - 29 November 2006