SCI-BUS gateway technology for a large variety of

SCI-BUS gateway technology for a large variety of user communities http: //www. sci-bus. eu Peter Kacsuk MTA SZTAKI SCI-BUS is supported by the FP 7 Capacities Programme under contract nr RI-283481 1

SCI-BUS EU FP 7 project • SCI-BUS (SCIence gateway Based User Support) provides gateway framework and science gateway building technology • 3 -year project: 1 Oct 2011 – 30 Sep 2014 • Objectives of SCI-BUS – Support both WF developers and end-user scientists – Create a generic-purpose science gateway framework – Elaborate a science gateway instance development technology – Establish production SG instance services both for national grids (horizontal user communities) and various science communities (vertical user communities) – Develop business models to guarantee sustainability and commercial exploitation 2

SCI-BUS Architecture 3

Community tools and user roles in SCI-BUS Data management Job submission Workflow Template A …. Data transfer Profile Monitoring Workflow Sequencing f. MRI simulator Non. Lin. Syst simulator …. Portlets Repository SCI-BUS Generic Framework based on WSPGRADE and Liferay Portlet Developers Gateway Developers Scientists Domain specific Gateway Portlet Function Portlet SCI-BUS Generic Framework Gateway Operators Application Function Applications (e. g. SHIWA) repositories Portlet DCI Application Developers DCI 4

Project partners No Participant organisation name Part. short name Country Expertise & Experience 1 Magyar Tudomanyos Akademia Szamitastechnikai es Automatizalasi Kutato Intezet MTA SZTAKI Hungary Coordinator in EDGe. S, EDGI, SHIWA, developer of WS-PGRADE 2 Academisch Medisch Centrum bij de Universiteit van Amsterdam AMC The Netherlands Gateway developer for the medical user community 3 Simsoft Bilgisayar Teknolojileri Ltd. Sti. SIMSOFT Turkey Gateway developer for business process modeling 4 E-GROUP Ict Software Informatikai Zrt EG Hungary Gateway developer for Web 2. 0 5 Eidgenössische Technische Hochschule Zürich ETH Zurich Switzerland Gateway developer for bioscience 6 Middle East Technical University METU Turkey Gateway developer for seizmology 7 Scaletools Ltd. ST Switzerland Gateway developer for clouds 8 Eberhard Karls Universität Tübingen EKUT Germany Gateway developer for computational chemistry 9 University of Westminster Uo. W UK Developer of UK NGS P-GRADE 10 Universidad de Zaragoza Unizar Spain Gateway developer for Pire. Grid SMEs 11 Cloud. Broker Gm. Bh CB Switzerland Gateway developer for clouds 12 4 D SOFT SZAMITASTECHNIKAI KFT 4 D SOFT Hungary Gateway developer for ETICS-2 13 Istituto Nazionale di Astrofisica INAF Italy Gateway developer for astrophysics 14 Laurea-ammattikorkeakoulu oy Laurea Finnland Gateway developer for Blender 15 The Provost Fellows & Scholars of the College of the Holy and Undivided Trinity of Queen Elizabeth near Dublin TCD Ireland Gateway developer for helio 5 physics

WS-PGRADE/g. USE Generic-purpose gateway framework • Based on Liferay • WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) • g. USE (Grid User Support Environment) architecture • General purpose • Workflow-oriented portal framework • Supports the development and execution of parameter sweep, Monte-Carlo applications • Enables the multi-DCI execution of any WF • Support the fast development of SG instances by a customization technology 6

Scalable architecture based on collaborating services

Flexibility of using various DCIs by WS-PGRADE/g. USE • Seamless access to various types of DCIs: • Clusters (PBS, LSF, MOAB, SGE) • Cluster grids (ARC, g. Lite, GT 2, GT 4, GT 5, UNICORE) • Supercomputers (e. g. via UNICORE) • Desktop grids (BOINC) • Clouds (Via Cloud. Broker Platform) 8

Integrated WS-PGRADE/Cloud. Broker Platform to access multi-clouds WSPGRADE 1 WSPGRADE n SEQ Cloud Broker Platform • Supported clouds: Amazon, Open. Stack, Eucalyptus, Open. Nebula, IBM • Saa. S solution: Iaa. S Cloud 1 Multi-cloud SEQ Iaa. S Cloud N • Preregistered services/jobs can run from WS-PGRADE Supported from g. USE 3. 5. 0 • Iaa. S solution: • any services/jobs (e. g. Bo. T jobs) can run from WSPGRADE Supported from g. USE 3. 5. 1 9

WS-PGRADE UI to configure nodes for cloud services Cloud selection During configuration WS-PGRADE queries the software list 10

Flexible usage scenarios/business models by WS-PGRADE/g. USE • Workflow developer view and support (full gateway framework view) • End-user view and support (limited portlets) • Customized user interface to support the creation of domain specific gateways (ASM API) • Provide workflow execution service on top of many different DCIs (Remote API) 11

Gateway types based on WS-PGRADE/g. USE Generic WS-PGRADE/g. USE gateway For developing workflow (WF) applications for a large set of various DCIs Developed WF Internal g. USE Repository External SHIWA Repository Existing Community Gateway Remote API End User mode WSPGRADE/g. USE gateway generated by configuration Customized g. USE gateway generated by ASM API WF execution g. USE gateway 12

The flexibility of using WSPGRADE/g. USE • Security: Flexible management of – Individual users’ certificate – Robot certificates • Data bridge (coming feature) – to access data storages in different DCIs – To transfer files among different DCIs • Support for workflow interoperability based on SHIWA technology – CGI approach for integrating many different type of workflows as meta-workflows – Using other workflows available in IWIR format in the 13 SHIWA repository

CTA requirements met by the software • The vast majority of the requirements are met • Instead of listing those that are met rather we show those that are not currently met 14

CTA requirements not met by the software • UR-SG-0020 The Science Gateway must provide access to the data collections (Archive) resulting from the Science data collection and to the MC simulation results through a data selection application. – This will be developed according to the needs of CTA community. Similar solution is under development, for example, for the ag. INFRA project • UR-SG-0920 All data obtained by CTA observatory must be made public through an archive following a period of proprietary use. The proprietary period could be different for each observation data set. – The portal is already connected to the public SHIWA Workflow Repository. Connecting to a public archive cannot be a problem. 15

Support for workflow developers in collaboration with ER-flow based on SHIWA technology SHIWA WF App. Repository Sharing WF appls via the repository • The SG framework provides graphical WF editor, WF manager • The SG framework provides access to a large set of clouds (and DCIs) to make these WF applications run Cloud 1 Science Gateway (SG) framework DCI (Grid, etc. ) Cloud N 16

Support for science gateway instance developers SHIWA App. Repository Science Gateway (SG) framework with access to various DCIs (including clouds) and access to SHIWA App. Repository Customization Science Gateway (SG) Instance with inherited access to App. Repo, clouds and DCIs (WS-PGRADE/g. USE) Cloud 1 Cloud N 17 different SG instances for biologists, chemists, etc. 17

Types of gateways to be developed from the core WS-PGRADE/g. USE framework 1. Generic purpose gateways for national grids – Core WS-PGRADE/g. USE (e. g. Greek, Italian NGIs) 2. Generic purpose gateway for a particular DCI – EDGI gateway based on WS-PGRADE/g. USE 3. Generic purpose gateway for specific technologies – SHIWA gateway for workflow sharing and interoperation 4. Domain specific science gateway instances – Swiss proteomics portal, Mo. SGrid gateway – Autodock gateway, Seizmology gateway, Viz. IVO, etc. 18

Communities developing WSPGRADE/g. USE based gateways • • 11 partner communities 6 subcontractors 5 associated members EU projects – – – ag. INFRA, DRIHM, VIALACTEA, EDGI, IDGF-SP, SHIWA, ER-Flow, Cloud. SME 19

g. USE based gateways 20

g. USE Roadmap based on community effort 21

g. USE download statistics at sourceforge 22

Where to find further information? • SCI-BUS web page: – http: //www. sci-bus. eu/ • g. USE/WS-PGRADE: – http: //www. guse. hu/ • g. USE on sourceforge – http: //sourceforge. net/projects/guse/forums/forum/ – http: //sourceforge. net/projects/guse/develop 23

24

Conclusions Join SCI-BUS as associated member Why to select WS-PGRADE/g. USE and join the SCI-BUS community? 1. Robustness – Already large number of gateways used in production 2. Sustainability – The sustainability and commercialization plan of the SCI-BUS project guarantees it, as well as the Cloud. SME project 3. Functionalities – Rich functionalities that are growing according to the SCI-BUS and sourceforge community needs 4. How easy to adapt for the needs of the new user community? – Already large number of gateways customized from g. USE/WSPGRADE 5. You can influence the progress of WS-PGRADE/g. USE 25