World Data Center for Glaciology Boul Facilitating the

World Data Center for Glaciology, Boul Facilitating the international exchange of snow and ice d The IPY Data and Information Service—How do we get there? Mark A. Parsons IPY Data Policy and Management Sub-committee IPY Data and Information Service Electronic Geophysical Year IPY Data Workshop Cambridge, England 3 March 2006 IPYDIS; Mark A. Parsons, 3 March 2006

IPY 1 IPY 2 2 IGY (IPY 3) IPY 4 IPYDIS; Mark A. Parsons, 3 March 2006

What will IPY 4 bring? The Challenge! • Will researchers be able to find all the data relevant to their research and see relationships between data sets. Access • Will they be able to merge and integrate different data sets across experiments and disciplines? Interoperability • Will they be able to subset, visualize, and transform the data? Usability • Will they be able to retrieve and understand IPY 4 data in 2050? Preservation 3 IPYDIS; Mark A. Parsons, 3 March 2006 IPY 4

Organization of IPY Data Management Data Policy & Management Subcommittee • scientists • data managers • funding agencies IPY Joint Committee Programme Office e. GY Data & Information Service Users Projects 4 Data Centers, Virtual Observatories, etc. IPYDIS; Mark A. Parsons, 3 March 2006

5 IPYDIS; Mark A. Parsons, 3 March 2006

Alternate Views of the DIS? 6 IPYDIS; Mark A. Parsons, 3 March 2006

Systems and Innovation “We're entering a new world in which data may be more important than software. ” - Tim O'Reilly Succeeded “Challenged” Failed The Standish Group’s “CHAOS report”. An assessment of over 40, 000 IT application projects 7 IPYDIS; Mark A. Parsons, 3 March 2006

The People Part “A striking proportion of project difficulties stem from people in both customer and supplier organisations failing to implement known best practice. ” — Oxford University/Computer Weekly survey of public and private sector IT projects (emphasis added) However, people are much more able to adapt to change, uncertainty, and messy systems Service counts. 8 IPYDIS; Mark A. Parsons, 3 March 2006

The People Part: Science and Data Management • Many have stated the need to involve scientists in data management, but… • It is also important to involve data managers in conducting science. • Field Experiments: 20% increase in data quality (Parsons, et al. 2004) • 70% of experiment cost is data assembly (Bernhardsen 1992, Longley, et al. 2001) • • Observing systems 9 IPYDIS; Mark A. Parsons, 3 March 2006

Preservation and Access—Two Peas in a Pod Scientific Data Stewardship: • “preservation and responsive supply of reliable and comprehensive data, products, and information for use in building new knowledge to…” —US Global Climate Research Program, 1998 • “the long-term preservation of the scientific integrity, monitoring and improving the quality, and the extraction of further knowledge from the data” — H. Diamond et al. , NOAA/NESDIS, 2003 10 IPYDIS; Mark A. Parsons, 3 March 2006

Access. What is it? “Facts are terrible things if left sprawling and unattended…” - Norman Cousins • Preservation requirements are well defined in the Open Archive Information System (OAIS) Reference Model, but • No similar model for access requirements • Not even a common definition of “access” and what restricts it • Unique access requirements for social science data and non-digital collections (physical samples, photographs, audio, etc. ) 11 IPYDIS; Mark A. Parsons, 3 March 2006

Standards—Essential but Cumbersome • Some Possibilities: ISO 19115 metadata standard OAIS Reference Model OGC data transfer standards Other OGC Standards “Web Services” (WSDL, SOAP) Other XML-based standards (GML, OAI-PMH, RSS, …) “We must not … start from any and • Etc, etc, • • No New Standards! 12 every accepted opinion, but only from those we have defined — those accepted by our judges or by those whose authority they recognize. ” —Aristotle c. 350 BC IPYDIS; Mark A. Parsons, 3 March 2006

01100010100100111101011 10001111011001010100011 Issues with the Data Itself 10011100101010011101010 100111000110100001 in “We often get blinded by the forms which content is produced, rather 000010010101100 than the job that the content does. ” 1001010100101 - Tim O’Reilly Formats: 0101010010100101 • Archives and users may have different 01000001111100101101010 101101000101111010 needs 1101001100010100100 • Consider four themes (Raymond, 2004) 11110101110001111011001 • Transparency 010100011100101010 • Interoperability 011101001110001101000010010 • Extensibility 0101011001001010100 • Storage or transaction economy 100100101010010101000001111100 1011010110100010111 101011 13 IPYDIS; Mark A. Parsons, 3 March 2006

Other Questions and Issues • How interoperable can we be? What does “portal” mean to you? • How do maximize use of existing data systems and structures? CODATA? WDCs? • How does IPY data fit into current operational systems? • What about GEOSS—can IPY be a prototype? • Which technological trends can help us? (ontologies, virtual observatories, portals, etc. ) • How do we incorporate historical data? • Need a solid business model esp. for the long-term 14 IPYDIS; Mark A. Parsons, 3 March 2006

Breakout Groups Methods for Data discovery—portals Paul Berkman, room 370 Ensuring data submission and publicatio-carrots and sticks Jim Moore, room 303 b Semantics, ontologies, and language Heather Lane, main room 15 IPYDIS; Mark A. Parsons, 3 March 2006

Charge to Breakout Groups 1. 2. 3. 4. 5. 6. 7. Determine rapporteur Explicitly define problem(s) Identify options to solve problem Recommend steps to solve problem Present to whole group for feedback Revise Write up results to be part of a larger workshop report. Include outstanding issues, next steps, etc. Workshop report will be presented to broader IPY research community for feedback and buy in. 16 IPYDIS; Mark A. Parsons, 3 March 2006