Under the hood 3500 source files in

Under the hood • 3500+ source files in java • J 2 SE, Tomcat, Pellet, My. SQL, Spring • 851 MB code base

How come? Date Instrument Parameter

Ambitious that I am …

Open-source Project for a Network Data Access Protocol test site

Things are not that easy…

Scientific Data Portal Modeling what exactly is a scientific data portal and how can we generate one? Linyun Fu Computer Science 2015 Advisor: Peter Fox 2013 -04 -30

Anything we could reuse? Modules, themes, templates, nodes, content types…

How did we build our portals? We just built them from scratch

NOAA Operational Model Archive Distribution System [Alpert AGU 2009]

http: //coreref. org/projects/and 1 -1 b/viewer/ [Reed AGU 2009]

http: //databasin. org/search/ [Comendant AGU 2009]

Arc. GIS layer

[Meyer AGU 2009]

Gulf of Mexico Coastal Ocean Observing System Information for Boaters, Fishermen and Divers GCOOS is a Regional Association of the U. S. Integrated Ocean Observing System. Members represent the private sector, government agencies, academia, and education and outreach sectors. GCOOS is concerned with sustained observations, integration of existing observing systems, sharing of non-proprietary data, and developing products and services. BOATING FISHING Information on bathymetry, water temperature, chlorophyll, ocean currents, salinity and dissolved oxygen can be used to locate favorable habitats and frontal systems where food animals aggregate. Information on waves, tides, currents, wind speed and direction, and air and water temperature aid navigation and improve marine forecasts. DIVING Information on surface and subsurface currents, temperature throughout the water column, water quality (e. g. red tide conditions) and water clarity can aid planning efforts and promote safer, more enjoyable diving. Multibeam map of the Pinnacles Region, Gulf of Mexico, showing a drowned reef complex along the 70 m isobath. Image credit: James Gardner, Peter Dartnell and Kenneth Sulak, USGS. Wave height and wave period data from WAVCIS Satellite false-color image showing chlorophyll (ug/l) distribution in the Gulf of Mexico. Image credit: USF Institute of Marine Remote Sensing. Poster credit: Christina Simoniello, USM Gulf Coast Research Laboratory www. gcoos. org Ocean current data showing profiles between the surfa Courtesy of Bob Weisberg, USF COMP

Then, someone said “Let’s reuse” Interoperability and reuse

INTEROPERABILITY - DEFINITION in·ter·op·er·a·bil·i·ty: ability of a system (as a weapons system) to work with or use the parts or equipment of another system. – Merriam-Webster's Collegiate Dictionary, Eleventh Edition. Between system and system. Ability to work together. Ability to share resources. As users of the Data Systems, we are more aware of and concerned with interoperability among data products. Versatile tools and standardized frameworks [Kuo AGU 2010]

[Truslove AGU 2011]

But we say “Why not generate? ” After we understand data portals

So a data portal is… Task fulfills User Interface presents Service supplies Dataset

How to use the model? • As a checklist of design issues for portal engineers • To compile best practices into a wellorganized recipe for quick reference – Sample dishes coming soon • To facilitate choice among frameworks • As the brain of the data portal generator – Inspiring ideas to follow…

ELDA system architecture

Service description spec: api a api: API; . . . api: sparql. Endpoint ; #api: sparql. Endpoint api: base "http: //aquarius. tw. rpi. edu: 8047/elda/cmspv"; . . . api: variable [api: name "base"; api: value "http: //cmspv. tw. rpi. edu/rdf"], . . .

URI template spec: vocabularies. Endpoint a api: List. Endpoint; api: uri. Template "/vocabs"; api: example. Request. Path "/vocabs"; api: selector [ api: where "? item rdf: type skos: Concept. Scheme. ? item skos: pref. Label ? label. "; api: order. By "? label"; ]; .

Default viewer and formatter http: //cmspv. tw. rpi. edu/rdf/vocabs

But we want more • More data source choices • Easier specification syntax – Or specification editor/wizard • More flexible data formats • Visualization and analysis support

GRASS GIS functionalities Geographic Resources Analysis Support System [Neteler 2008]

nviz

Py. WPS Web Processing Service implemented in Python

Vis. Ko Open Source Visualization Knowledge System [Del Rio 2012]

A sample Vis. Ko query VISUALIZE http: //rio. cs. utep. edu/ciserver/ciprojects/Gravity. Map. Prov enance/gravity. Dataset. txt AS views: 2 D_Contour. Map IN visko: mozilla-firefox WHERE FORMAT = formats: SPACESEPARATEDVALUES. owl#SPACESEPARATEDVALUES AND TYPE = types: d 19 AND C = 10 AND A = 20

Vis. Ko view ontology

IPython Notebook – interactive developing/testing

Roadmap for a portal generator • Content oriented portal software • Understand scientific data portals – Interoperability and reuse • Define portals with specification files • Embrace specialized tools – Open source – Open standards – High-level coding • Allow testing as you go

• Project page: http: //tw. rpi. edu/web/project/Se. SF/workinggroups/ Scientific. Data. Portal. Generator • Point of contact: Linyun Fu ful 2@rpi. edu

Selected references • • • VSTO: Fox, P. , Mc. Guinness, D. L. , Cinquini, L. , West, P. , Garcia, J. , Benedict, J. , and Middleton, D. 2009. Ontology-supported Scientific Data Frameworks: The Virtual Solar. Terrestrial Observatory Experience. Computers & Geosciences, pages 724– 738. OPe. NDAP: West, P. , et al. "OPe. NDAP Hyrax: An extensible data access framework within the Earth System Grid Federation. " AGU Fall Meeting Abstracts. Vol. 1. 2011. Semant. Eco: Wang, P. , Fu, L. , Patton, E. W. , Mc. Guinness, D. L. , Dein, J. , and Bristol, S. 2012. Towards Semantically-enabled Exploration and Analysis of Environmental Ecosystems. In Proceedings of 8 th IEEE International Conference on e. Science (October 812 2012, Chicago, IL). NOMADS: Alpert, J. C. , and J. Wang. "NOAA Operational Model Archive Distribution System (NOMADS): High Availability Applications for Reliable Real Time Access to Operational Model Data. " AGU Fall Meeting Abstracts. Vol. 1. 2009. ANDRILL MIS: Reed, J. , et al. "Web-based Collaboration and Visualization in the ANDRILL Program. " AGU Fall Meeting Abstracts. Vol. 1. 2009. Data Basin: Comendant, T. , et al. "Data Basin: Expanding Access to Conservation Data, Tools, and People. " AGU Fall Meeting Abstracts. Vol. 1. 2009.

• • • About user needs survey: Meyer, D. J. , and K. P. Gallo. "Enabling data access and interoperability at the EOS Land Processes Distributed Active Archive Center. " AGU Fall Meeting Abstracts. Vol. 1. 2009. GCOOPS: – Jochens, Ann. "Gulf of Mexico Coastal Ocean Observing System. " Ocean Views: News from the Ocean. US Office. Issue 29 (2006): 1. – Howard, M. K. , F. C. Gayanilo, and A. E. Jochens. "Regional Ocean Data Portal: Transforming Information to Knowledge. " AGU Fall Meeting Abstracts. Vol. 1. 2009. About interoperability: Kuo, K. "Interoperability Barriers in NASA Earth Science Data Systems from the Perspective of a Science User. " AGU Fall Meeting Abstracts. Vol. 1. 2010. About reuse: Truslove, I. , et al. "A Software Architecture To Encourage Internal And External Software Reuse. " AGU Fall Meeting Abstracts. Vol. 1. 2011. Cmap. Tools: A Knowledge Modeling and Sharing Environment, A. J. Cañas, G. Hill, R. Carff, N. Suri, J. Lott, T. Eskridge, G. Gómez, M. Arroyo, R. Carvajal (2004). In: Concept Maps: Theory, Methodology, Technology, Proceedings of the First International Conference on Concept Mapping, A. J. Cañas, J. D. Novak, and F. M. González, Editors , Universidad Pública de Navarra: Pamplona, Spain. p. 125 -133. Semantic Application Design Language (SADL): http: //sadl. sourceforge. net/

• ELDA: http: //elda. googlecode. com/hg/deliverelda/src/main/webapp/lda-assets/docs/E 1. 2. 19 -index. html • GRASS GIS: Neteler, Markus, and Helena Mitasova. "Open Source GIS: A GRASS GIS Approach. The International Series in Engineering and Computer Science. " (2008): 406. • Vis. Ko: Nicholas Del Rio and Paulo Pinheiro da Silva. Capturing and Using Knowledge about the Use of Visualization Toolkits. AAAI Fall Symposium on Discovery Informatics: The Role of AI Research in Innovating Scientific Processes, Arlington, Virginia, November 2, 2012. • IPython Notebook: https: //github. com/ipython/wiki/A-gallery-ofinteresting-IPython-Notebooks