P-GRADE Portal Jozsef Patvarczki Worcester Polytechnic Institute MA

P-GRADE Portal Jozsef Patvarczki Worcester Polytechnic Institute, MA patvarcz@cs. wpi. edu 1

E-scientists’ concerns • How to concentrate on my own research if the technology I would like to use is in continuous change? • Which is the most suitable grid for me? • How can I learn and understand the usage of that technology? • How can I develop applications? • How can I execute applications? • How to tackle performance issues? • How to use several Grids at the same time? • How to migrate my application from one grid to another? • How can I collaborate with fellow researchers? The P-GRADE Grid Portal gives you all the answers! 2

Easy-to-use, middleware-neutral workflow-oriented Grid portal • Graphical tools for workflow-oriented application development, execution and monitoring • Sequential & parallel components can be integrated into large Grid applications • Direct or broker-based resource allocation • The P-GRADE Portal hides differences of Grids • Globus 2 and 4 – LCG 2 – g. Lite Grid interoperability • Transparent switching between Grid technologies • Your code does not have to contain grid specific calls 3

P-GRADE portal in a nutshell Proxy management Grid and Grid resource management Graphical editor for workflow creation Job mapping to Grid and Grid resources Workflow management and execution visualization 4

What is a P-GRADE Portal workflow? • a directed acyclic graph where – Nodes represent jobs (batch programs to be executed on a computing element) – Ports represent input/output files the jobs expect/produce – Arcs represent file transfer operations and job dependencies • semantics of the workflow: – A job can be executed if all of its input files are available 5

Introducing three levels of parallelism Multiple instances of the same workflow with different data files – Parallel execution inside a workflow node – Parallel execution among workflow nodes – Parameter study execution of the workflow Multiple jobs run parallel Each job can be a parallel program 6

PS_1_2 PS_1_1 PS DATA FLOW PS_1_1 PS_2_1 PS_1_2 PS_2_1 PS_1_1 Store the results of the calculations (Results are represented by remote files of one or more common directories) PS_2_2 PS_2_1 PS_2_2 PS_1_2 One or more Parameter Sets are defined. Each has different parameters. (Sets are represented by remote file directories, parameters by files) PS_2_2 WF Result_1_1 Result_2_1 Result_1_2 Content of Parameter Sets are combined (Descartes product) The workflow is executed as many times as many elements are in the Descartes product of input sets Result_2_2 7

Summary of PS execution First phase: executing ones all the Generators Last phase: executing ones all the Collectors Second phase: executing all generated e. Workflows in parallel 8

Workflow Management Window showing Generator and Collector phase is inactive until each e. Workflow has terminated 9

Grid interoperability 1 • P-GRADE portal supports simultaneous job submission in several Grids: – GT 2 (OSG, Tera. Grid and UK NGS) – LCG-2, g. Lite – EGEE, SEEGRID – Nordu. Grid • P-GRADE Portal enables: – Connecting production Grids and make them interoperable at workflow level 10

Connecting various production Grids OSG (GT 2) Certificate server Proxy WPI Portal Server http: //pgrade. wpi. edu User Workflow Job P-GRADE Portal ANL Fermilab SEEGRID (g. Lite) Budapest Supports both direct and brokered job submission Job g. Lite broker Job Istambul Job Athens 11

SEEGRID: the “e. Infrastructure inclusion” into SEE South-East European Gridenabled e. Infrastructure • 11 Countries, Infrastructure overlapping and complementing to EGEE – SEEGRID open to new “fresher” sites – SEEGRID open to new applications SEER EN http: //www. see-grid. eu SEER EN EGEE SEE-GRID 12

Grid interoperability 2 • P-GRADE Portal with GEMLCA supports legacy code service invocation in 2 nd and 3 rd generation Grids: – GT 2, GT 4 – (OSG, Tera. Grid and UK NGS) – LCG-2, g. Lite – (EGEE, SEEGRID) • Combination of jobs and legacy code services enables GEMLCA/P-GRADE Portal to support: – Extending 2 nd generation Grids with legacy code services – Extending 2 nd generation Grids with GT 4 sites – Connecting 2 nd generation and 3 rd generation Grids 13

Connecting 2 nd generation and 3 rd generation production Grids OSG (GT 2) GIN Portal Server GEMLCA P-GRADE Portal User e bl cuta Exe Fermi Lab Executable EGEE LCG/g. Lite ANL Service Invocation. West. Focus GT 4 Executable Poznan Budapest Grid Executable Job Repository (GEMLCA) Uo. W Brunel 14

Molecular Dynamics Study of Water Penetration in Staphylococcal Nuclease using CHARMm • The Portal has been used by Ana Damjanović from Johns Hopkins University to submit adapted shell scripts for the running of CHARMm jobs for the Molecular Dynamics Study of Water Penetration in Staphylococcal Nuclease. • A typical CHARMm workflow would start with a heating phase followed by an equilibrium phase (each of these appx. 30 hours). • Simultaneous to the above, a pair of small jobs happen which prepare the common parameter files. • Followed by one or more simultaneous sequences of production runs (each production run appx. 90 hours) 15

CHARMm Workflow 16

Execution Visualisation 17

P-GRADE portal installations 18

GIN VO Portal 19

How to learn the P-GRADE portal? – Take a look at www. lpds. sztaki. hu/pgportal (manuals, slide shows, installation procedure, etc. ) – Visit or request a training event! (event list also on homepage) • Lectures, demos, hands-on tutorials, application development support – Get an account for one of its production installations: • • WPI – http: //pgrade. wpi. edu SEEGRID portal – SZTAKI GILDA portal – SZTAKI NGS portal – University of Westminster – If you are the administrator of a Grid/VO then contact SZTAKI to get your own P-GRADE Portal! – If you know the administrator of a P-GRADE Portal you can ask him/her to give access to your Grid through his/her portal installation! (Multi-Grid portal) 20

Learn once, use everywhere - portal Develop once, execute anywhere - application Thank you! patvarcz@cs. wpi. edu 21