New developments of g USE WS-PGRADE to

New developments of g. USE & WS-PGRADE to support e-science gateways Peter Kacsuk and Miklos Kozlovszky MTA SZTAKI - Computer and Automation Research Institute of the Hungarian Academy of Sciences EGI User Forum 2011 Vilnius, Lithuania 11. 04. 2011. www. lpds. sztaki. hu

Content 1. 2. 3. 4. WS-PGRADE & g. USE in a nutshell Basic and new services Case studies How to access 2

P-GRADE Portals around the globe • National Grid portals (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia , Kazakhstan, Switzerland, Australia, Cl. Grid, etc. ) • HP-SEE/SEE-GRID-SCI, Baltic. Grid • US Open Science Grid, Tera. Grid • IDGF (International Desktop Grid Federation) • SHIWA Simulation Platform • Economy-Grid, Swiss Bio. Grid, Bio és Biomed Virtual Organisations, Math. Grid, etc. HP-SEE 3

P-GRADE Portal software family P-GRADE Portal 2. 4 2008 P-GRADE Portal 2. 5 P-GRADE Portal 2. 8 GEMLCA Grid Legacy Code Arch. NGS P-GRADE portal Basic concepts WS-PGRADE Portal 3. 1 2009 P-GRADE Portal 2. 9. 1 2010 2011 GEMLCA, storage concepts P-GRADE Portal 2. 10 Open source from 2008 WS-PGRADE Portal 3. 2 stats: WS-PGRADE Portal 3. 3 4

Motivations of creating g. USE • To overcome (most of) the limitations of P-GRADE portal: • To provide better modularity to replace any service • To improve scalability to millions of jobs • To enable advanced dataflow patterns • To interface with wider range of resources • To separate Application Developer view from Application User view WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) and g. USE (Grid User Support Environment) architecture 5

g. USE and WS-PGRADE architecture Graphical User Interface: WS-PGRADE Workflow storage Workflow Engine Meta-broker g. USE information system Submitters Gridsphere LIFERAY Large set of portlets File storage Application repository Autonomous Services: high level middleware service layer Logging Local resources, service grid VOs, Desktop Grid resources, Web services, Databases Resources: middleware service layer 6

WS-PGRADE/ g. USE • Creating complex workflow and parameter sweeps • Seamless access to various types of resources • clusters, • service grids, • desktop grids, • databases. • Scalable architecture • Advanced data-flows and workflow concept • Creating complex applications using embedded workflows, legacy codes • Comfort features • Separated views for application developers and end-users • Community components from workflow repository www. wspgrade. hu 7

WS-PGRADE in a nutshell • General purpose, workflow-oriented portal. Supports the development and execution of workflow-based applications • Based on Liferay • Services supported by the portal: • New functionalities Basic Services EGEE grids Globus grids Desktop clusters (LCG 2, GLite grids (GT 2, GT 4) • Web services ) • DB connectors Job execution File storage • Embedded workflows • Job level PS Certificate management Information system • Conditional jobs Brokering • Recursive graph Job monitoring • Multi-generator Workflow & job visualization • Multi-collector • CROSS product PS • DOT product PS Simultaneous use of various clusters, service and desktop Grids at workflow level 8

Latest developments of g. USE • g. USE service orchestration • direct remote access • ASM (Application Specific Module) • JAVA API • New submitters • ARC • UNICORE + SAML (Security Assertion Markup Language) support • LFS support 9

g. USE service orchestration Accessing g. USE services remotely • direct remote access • ASM (Application Specific Module) • JAVA API WS-PGRADE MOSGRID/x ASM Remote access API g. USE Prog. Env. (JAVA API) Remote access module (JAVA API) Chemaxon Ltd. Remote client (CLI) SHIWA 10

WS-PGRADE in European and national projects • Cancer. Grid (EU FP 6) • Pro. Sim (UK) • EDGe. S and EDGI (EU FP 7) • Mos. Grid (D-Grid) • Swiss Proteomics Portal • HP-SEE (EU FP 7) • SHIWA (EU FP 7) • SCI-BUS (EU FP 7) 11

The Cancer. Grid portal for anti-cancer drug research Workflow execution Workflow development & configuration Algorithms configuration Integrated components of Cancer. Grid portal Molecule database browser Structure viewer 12

Pro. Sim workflow in g-USE • Phase 1 UK project for Protein Molecule Simulation on the Grid Phase 2 • a combination of GEMLCA and standard g-USE jobs • Executed on 5 different sites of the UK NGS • Parameter sweeps in phases 3 and 4 • Status: actively used in production 13

Mos. Grid Science Gateway for Molecular Simulations • Requirement: Workflow enabled grid portal for UNICORE based on Liferay • Solution: • Adaptation of WS-PGRADE from Grid. Sphere to Liferay • Development of UNICORE Submitter • Support for SAML • Development of ASM support • Status: • It’s prototyped and ~15 community members tested • Planned production provision from May 2011 • See details: • Previous presentation by Sandra Gesing: A Science Gateway for Molecular Simulations 14

Swiss Proteomics Portal • TPP Workflow INPUT parameters. properties Param. A = A 1; A 2 Param. B = B 1; B 2; B 3 Param. C = C 1 file_list. txt mz. XML_1 mz. XML_2 mz. XML_3 combine Parameter Sets Result PS 1&mz. XML Result_1 PS 1&mz. XML _2 Result _1 PS 2&mz. XML Result_1 _2 PS 2&mz. XML Result _1 PS 3&mz. XML Result_1 _2 PS 3&mz. XML Result _1 PS 4&mz. XML Result_1 _2 PS 4&mz. XML Result _1 PS 5&mz. XML Result_1 _2 PS 5&mz. XML Result _1 PS 6&mz. XML Result_1 _2 PS 6&mz. XML _1 PS 2 Param. A = A 2 Param. BPS 3 = B 3 Param. BPS 4 = B 2 Param. C = C 1 A 2 Param. A = Param. C = C 1 A 1 Param. A = Param. B = B 1 PS 5 Param. B = B 3 Param. C = C 1 Param. A =Param. C = C 1 A 1 PS 6 Param. B = B 2 Param. A Param. C = C 1 = A 1 Param. B = B 1 Param. C = C 1 Data Set mz. XML_1 mz. XML_2 mz. XML_3 Status: Production since Dec 2010 in ETH Zürich See details: Previous presentation: Proteomics portal: an e. Science Gateway for Swiss Bioinformatics Step 1 OUTPUT Result PS 1 Result PS 2 Result PS 3 Result PS 4 Result PS 5 Result PS 6 Step 2 19/05/10 15

SHIWA Simulation Platform v 1 (P-GRADE) and v 2 (WS-PGRADE) Creating multi-workflow applications and executing them in the multi. SHIWA VO middleware SHIWA VO SHIWA Science Gateway SHIWA Portal local cluster SHIWA Repository WF 1 WFn WF list WEp GEMLCA Repository PGRADE Workflow editor MOTEUR WE WFm WE 1 s 2 WE + WF ASKALON WE Kepler WE edit WF s 2 ASKALON WE s 4 MOTEUR WE GWES WE e-scientist s 3 PGRADE Workflow engine Taverna WE s 1 WF 1 Globus DCI Triana WE search WF GEMLCA admin g. Lite DCI GEMLCA with GIB GEMLCA Service s 5 invoke WE pre-deployed. WEs 16

Portal Access • Portal related information • www. wspgrade. hu • Access to multi-grid environment • WS-PGRADE/ g. USE portal • https: //guse. sztaki. hu/liferay-portal 6. 0. 5/ GASUC • For application porting • GASUC (Grid Appl. Support Centre) success stories • http: // www. lpds. sztaki. hu/gasuc / • Interoperable workflow and DCI support • SHIWA Simulation platform • http: //www. shiwa-workflow. eu 18

Invitation to related events • PUCOWO (P-GRADE User COmmunity WOrkshop), London, 7 -8 June 2011 http: //portal. p-grade. hu/pucowo 2011/ • IWSG-Life (International Workshop on Science Gateways for Life Sciences), London, 8 -18 June 2011 https: //sites. google. com/a/staff. westminster. ac. uk/iwsg-life 2011/ • DCI Summer School: Grid and Cloud for e-science Budapest, 11 -17 July 2011 http: //www. lpds. sztaki. hu/eudciss 2011/ 19

Thank you for your attention! Questions? kacsuk@sztaki. hu www. lpds. sztaki. hu 20