NODCi — National Infrastructure for access to Oceanographic

NODCi - National Infrastructure for access to Oceanographic and Marine Data and Information (RGI-014) By Dick M. A. Schaap – MARIS Amsterdam, April 09 1

NODCi project summary • implementation of National infrastructure for access to oceanographic and marine data and information • must be interoperable with international networks and developments • phase 1: definition (early 2005 – march 2006) • phase 2: implementation (mid 2006 – march 2009) • partners: – KNMI – TNO - Geological Survey – RWS – NIOO-CEME – NIOZ – Hydrographic Office – Deltares – MARIS (coordinator) • co-funding by Ruimte voor Geo-Informatie (RGI) programme (RGI-014) 2

International developments • Sea-Search: Pan-European network for oceanographic and marine data management (2002 – 2005) – focus on metadata • Sea. Data. Net - Pan-European network for oceanographic and marine data management (2006 – 2011) – continuation of metadata and access to data • EU - INSPIRE Directive – European Spatial Data Infrastructure • GEO-GEOSS initiative from the G 8 ad-hoc Group of Earth Observations for a Global Earth Observation System of Systems • GMES (Global Monitoring for Environment and Security) from ESA and EU Research as European component of GEOSS • New EU - Marine Directive with plans for an EMODNET • UNESCO – IOC – IODE – Intergovernmental Oceanographic Commission • ICES – International Council for Exploration of the Seas 3

NODCi approach • Cooperating and tuning with the projects Sea-Search and its successor Sea. Data. Net for standards and technology • NODCi as Netherlands node in the Sea-Search / Sea. Data. Net infrastructure • Tuning with INSPIRE developments via European contributions with Sea. Data. Net as Spatial Data Interest Community (SDIC) and nationally via cooperation with RAVI and later Geo. Novum • Exchanging progress and tuning with IOC-IODE, ICES, WMO, EGS and other international groups and fora 4

NODCi activities • establishing a new NODCi portal connected to data management systems of individual NODC members • populating the Sea-Search V 0 and successive Sea. Data. Net V 1 metadata directories • establishing harmonised and transparant online access to the data collections, managed by the NODC members, in line with Sea. Data. Net • establishing a pilot application service with a numerical model • establishing OGC compliant viewing services for spatial information for the Netherlands part of the North Sea • preparing an official arrangement with NODC members for long-term sustainability and operation of the NODCi infrastructure • dissemination, promotion and regular dialogue with potential users from government, research and industry 5

More about Sea. Data. Net aims to set up and operate an efficient Pan-European distributed infrastructure for managing marine and ocean data by connecting: 40 National Oceanographic Data Centres (NODC’s), national oceanographic focal points, and ocean satellite data centres, in Europe these Data Centres are mostly divisions of major national marine research institutes and based in 35 countries, surrounding the European seas Sea. Data. Net aims to ensure a comparable quality of data sets and to make data sets easily accessible on-line through a unique portal, while the data sets are stored and managed at the Data Centres. 6

Sea. Data. Net Participating Countries 49 Partners = 40 Data Centres + ICES + IOC-IODE + JRC + specialists in statistics, products and informatics 7

Sea. Data. Net Infrastructure overall concept 8

Sea. Data. Net Infrastructure versions Version 0 = Continuation and maintenance of existing Sea-Search systems and population of the Common Data Index (CDI) V 0 system Version 1 = Harmonised and upgraded metadatabases adopting ISO 19115 standard; Transparent data access involving all Data Centres via implementation of Common Data Index (CDI) V 1 system and downloading services. Version 2 = Adding OGC viewing services and further virtualisation of data access. 9

Sea. Data. Net Services Sea. Data. Net portal (www. seadatanet. org) www. seadatanet. org Pan-European metadata directories: EDMED - European Directory of Marine Environmental Data Sets EDMERP – European Directory of Marine Research Projects CSR – Cruise Summary Reports EDIOS – European Directory of Ocean observing Systems EDMO – European Directory of Marine Organisations Common Data Index (CDI) for detailed overview and harmonised access to data sets, managed by Sea. Data. Net data centres Generic regional data products, available by viewing services 10

Sea. Data. Net Interoperability is the key to distributed data management system success. This is achieved via: Standardised metadata formats, based upon ISO 19115, supported by controlled Vocabularies Standardised QC methods and Quality Flags, complying with international standards Standard software tools and protocols for interconnecting the Data Centres and the Portal Portable data management software to equip the Data Centres for file conversion, analysis, and viewing Standard formats for delivering data sets to users Common geostatistics software (DIVA) for computing regional gridded fields and statistics Web services as key technology 11

Sea. Data. Net Common Vocabularies User Client Interface http: //seadatanet. maris 2. nl/v_bodc_vocab/welcome. aspx 12

Sea. Data. Net Data policy Metadata free and open access, no registration required each data centre is obliged to provide the meta-data in standardized format to populate the catalogue services Data and products services the general case is free and open access however user must register once in order to get a personal login password user agrees with “Sea. Data. Net User Licence” licence is part of the Sea. Data. Net Data Policy, that is fully compatible with the Directive of the European Parliament and of the Council on public access to environmental information, the INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR data principles. Sea. Data. Net Data Policy is an overarching policy, respecting any local policy Access to data is regulated by the User role and the specified Data Access restriction 13

Sea. Data. Net V 1 ODV and DIVA presentation and analysis tools 14

NODCi activities • establishing a new NODCi portal • populating the Sea-Search V 0 and successive Sea. Data. Net V 1 metadata directories • establishing harmonised and transparant online access to the data collections, managed by the NODC members, by connecting the data management systems of individual NODC members to the portal • establishing a pilot application service with a numerical model • establishing OGC compliant viewing services for spatial information for the Netherlands part of the North Sea • preparing an official arrangement with NODC members for long-term sustainability and operation of the NODCi infrastructure • dissemination, promotion and regular dialogue with potential users from government, research and industry 15

NODCi portal http: //www. nodc. nl 16

Access to data - Common Data Index (CDI) CDI is a fine-grained index to individual data measurements (such as a CTD cast, moored instrument record, sealevel station timeseries) CDI XML Schema is based upon the ISO 19115 standard CDI V 0 developed in Sea-Search; upgraded to V 1 in Sea. Data. Net CDI V 0 = Central and homogeneous metadatabase, but for Data Access users are confronted with the different user interfaces of the NODC Data Centres (which can provide online search and access / ordering facility / e-mail request) 17

NODCi CDI V 0 – User Interface 18

NODCi CDI V 0 – Data Coverage CDI V 0 coverage: ca. 42. 000 CDI entries from 5 NODC members 19

NODCi CDI V 0 – Data Coverage CDI V 0 coverage: TNO collection = > 10. 000 CDI entries for geological samples and cores 20

Common Data Index (CDI) V 1 CDI V 0 XML Schema has been upgraded to V 1 XML Schema, supported by Common Vocabularies, editing/generation tool (MIKADO) and upgraded XML validation services CDI V 1 has been adopted in NODCi phase 2 to give users not only a homogeneous metadatabase of available data, but also a unique interface for getting access to the data sets themselves, that are managed by the distributed NODC members This includes connecting the NODCi CDI portal with the data management systems of the NODC members by adopting tools and principles, developed in Sea. Data. Net The NODCi implementation served also as learning curve and provided Sea. Data. Net very useful feedback, which has lead to various improvements and additional functionality 21

Delivery Services – CDI V 1 Central and homogeneous metadatabase plus transparent access to the distributed data sets via a unique user interface at the NODCi portal and downloading services 22

Data Access components Common Data Index (CDI) system including a Shopping and Transaction mechanism Using harmonised Data Transport Formats for data sets delivery Authorisation and Authentication Services, based upon the Central User Register and the Data Access Restrictions as given in the CDI metadata 23

CDI V 1 system components CDI V 1 Metadatabase and User Interface: Searching and browsing of metadata of data sets, managed by Data Centres Central User Register - Web service: Details of users, their organizations and addresses, license period, Id-Passwords, Roles Shopping Basket – User Interface: Preparing a user request of multiple data sets, handling validation of users and roles via the Central User Register, and routing requests to the Request Status Manager – Web service + User Interface: Processing and administration of all requests and data deliveries (downloads), for users in communication with data centres. For data centres to oversee all transactions. 24

CDI V 1 system components Data Centre Download Manager – Java Tool: Handles communication with Request Status Manager and takes care that requested files are made ready for downloading by User (if OK) in the agreed formats via a local website address at the Data Centre 25

NODCi CDI V 1 – User Interface Many options, but difficult for first-time users 26

NODCi CDI V 1 – Quick Search User Interface Easy, because very intuitive 27

NODCi CDI V 1 – Data Access example Check Status In RSM Search Include in Basket Request Confirmed Results Ready at DC x Shopping list Download Submit + Authentication Data SDN format 28

Pilot application service NODCi GIS Viewer, combining North Sea topographic and bathymetric maps from RWS and TNO via OGC-WMS Viewer is based upon Minnesota Map Server Viewer includes layers for oil & gas licences, pipelines and platforms Viewer expanded with a demonstrator for an oil-spill simulation model, based upon Deltares ‘Delft 3 D-Part User can calculate and visualise oil spills Oil spill model results are exported in Net. CDF format and stored at Deltares on nc. WMS server NODCi GIS Viewer enables to select model runs and to include the WMS output as a layer in the GIS Viewer 29

NODCi GIS Viewer 30

NODCi Oil Spill entry form 31

Cross sector involvement Sea. Data. Net cooperates and provides services for many other European and international groups and projects, contributing and safeguarding good data management. Examples: Euro. GOOS, maintaining the European Directory of Ocean observing systems (EDIOS) and improving access to real-time data. See www. edios. org POGO, collecting and providing information on ocean-going research vessels, and their operators, planned and completed cruises. See www. pogo-oceancruises. org GMES Marine Core Services for streamlining the provision of long-term archives for optimising marine forecast services Euro. Geo. Surveys for improving overview and access to geological and geophysical data sets See www. eu-seased. net Partners in the Humboldt project (GMES) contributing to a European Spatial Data Infrastructure Services for the international oil & gas industry for metocean data (SIMORC). See www. simorc. org Associated infrastructure projects for the Black Sea and Caspian Sea communities (Black. Sea. SCENE and CASPINFO) 32

EMODNET perspective New Marine Directive includes the establishment of an overarching European Marine Observations and Data Network – EMODNET will integrate existing and new to be developed observation networks and data management systems into a coherent Pan-European network for providing easy discovery and access of data and derived products to users from science, industry and general public As part of EMODNET development recent 4 data pilot tenders have been set out by the EU EMODNET Taskforce for hydrography, geology, chemicals and biology Sea. Data. Net subgroups have been successful in their application and are awarded the projects execution NODC as partner involved in the pilot for chemical data, that will be developed using the Sea. Data. Net (and NODC infrastructure) in close cooperation with the Marine Conventions (OSPAR for North Sea – Atlantic Ocean) and EEA (Marine WISE and SEIS) 33

NODCi – FES proposal New national funding programme – FES NODC has applied for further development of the NODCi infrastructure: Expanding number of connected data holding institutes with IMARES and the Hydrographic Office Including socio-economic data sets Additional functionalities for visualisation, presentation and analyses of data sets Developing and testing support and analysis tools for a number of use cases, such as planning of new windfarms 34

Demonstration 35