Enabling Grids for E-scienc E Application specific portlet

Enabling Grids for E-scienc. E Application specific portlet other portal features and next steps Miklos Kozlovszky MTA SZTAKI m. kozlovszky@sztaki. hu www. eu-egee. org EGEE-II INFSO-RI-031688

Complete P-GRADE development cycle Enabling Grids for E-scienc. E • Develop your jobs locally • Compile your jobs for the grid – Compiler portlet • Compose workflows from your jobs and execute it – Workflow editor, workflow manager, Information system portlets • Scale up the workflow to a parameter study – Define input files (or add Input file generator) – Add output file collector • Develop an application specific portlet: –To hide application logic from end users: Separate grid experts from grid users EGEE-II INFSO-RI-031688 2

P-GRADE and user roles Enabling Grids for E-scienc. E Portal Administrator: • installs P-GRADE Portal 2. 7 Grid Application Developer • develops a grid application with P-GRADE Portal 2. 7 • sends the application to the grid portal developer Grid Portal Developer • Creates new classes from the Application Specific Module for P-GRADE by changing the names of the classes • develops a Gridsphere portlet that fits to the application I/O pattern and the end users’ needs • connects the GUI to P-GRADE Portal using the programming API of P-GRADE Application Specific Module • using the Application Specific Module he publishes the grid application and its GUI for end users End Users • executes the published application with custom input parameters by creating application instances using the published application as a template EGEE-II INFSO-RI-031688 3

1. 2 Basic idea Enabling Grids for E-scienc. E • A new module called “Application specific module” has been developed for P-GRADE Portal that ease the creation of Application Specified Portlets – The module provides programming API to control P-GRADE portal applications – The module provides programming API to access P-GRADE Portal services • Any kind of Graphical interface can be developed on top of this module – GUI written in JSP, JSTL (Java Servlet Tag Library ) and Java – GUI can be customized for the grid application – GUI can be customized for end users – GUI can benefit from all the P-GRADE Portal services EGEE-II INFSO-RI-031688 4

1. 3 Concept Enabling Grids for E-scienc. E Client Web browser You develop it Custom user interface Additional module for P-GRADE Portal server (Written in Java, JSP, JSTL) Application specific module Core P-GRADE Portal Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) Grid EGEE-II INFSO-RI-031688 EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, …) 5

Detailed Scenario Enabling Grids for E-scienc. E Portal Administrator installs the P-GRADE portal 2. 7. Then adjusts the portlet to publish applications Publisher Part of Application specific module Portlet to publish applications Registry of published applications Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) P-GRADE PORTAL 2. 7 server EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, Grid. FTP, …) EGEE-II INFSO-RI-031688 6

Detailed Scenario Enabling Grids for E-scienc. E Grid Application Developer creates own Grid Application. Publisher Part of Application specific module Grid Application Portlet to publish applications Registry of published applications Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) P-GRADE PORTAL 2. 7 server EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, Grid. FTP, …) EGEE-II INFSO-RI-031688 7

Detailed Scenario Enabling Grids for E-scienc. E Grid Portal Developer develops an Application specific portlet using Java Server Pages and Gridsphere Tag Library Updates the application specific package: connects Grid application to new portlet Publisher Part of Application specific module Grid Application specific portlet Portlet to publish applications Registry of published applications Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) P-GRADE PORTAL 2. 7 server EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, Grid. FTP, …) EGEE-II INFSO-RI-031688 8

Detailed Scenario Enabling Grids for E-scienc. E Grid Portal Developer develops an Application specific portlet using Java Server Pages and Gridsphere Tag Library Updates the application specific package: connects Grid application to new portlet Publisher Part of Application specific module Grid Application specific portlet Portlet to publish applications Grid Application (Template) Registry of published applications Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) P-GRADE PORTAL 2. 7 server EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, Grid. FTP, …) EGEE-II INFSO-RI-031688 9

Detailed Scenario Enabling Grids for E-scienc. E End users can create projects from a Published Grid Application Template. Then they can adjust parameters, and run in Grid via Application specific portlet! Publisher Part of Application specific module Grid Application specific portlet Grid Application (Template) Grid Application Portlet to publish applications Registry of published applications Services of P-GRADE Portal (workflow management, parameter study management, fault tolerance, …) P-GRADE PORTAL 2. 7 server EGEE and Globus Grid services (g. Lite WMS, LFC, …; Globus GRAM, Grid. FTP, …) EGEE-II INFSO-RI-031688 10

Interface to publish a P-GRADE Application Enabling Grids for E-scienc. E Grid Application Developer can publish the application by a pre-developed portlet Button to revoke an already published application Button to publish the application Select an application EGEE-II INFSO-RI-031688 11

Enabling Grids for E-scienc. E Example: Matrix multiplication on EGEE end users’ point of view Do you see the grid somewhere? www. eu-egee. org EGEE-II INFSO-RI-031688

Manage your computations Enabling Grids for E-scienc. E Button to load a multiplication project Button to delete a multiplication project Button to create new multiplication project EGEE-II INFSO-RI-031688 13

Define a new multiplication task Enabling Grids for E-scienc. E Name of the current project Matrix A (you can define values for Y) EGEE-II INFSO-RI-031688 Matrix B is fixed 14

Define values for Y Enabling Grids for E-scienc. E Parameters can be adjusted as Set, Range and Random. EGEE-II INFSO-RI-031688 15

Monitor progress Enabling Grids for E-scienc. E Click to start computation Status bar of computation Click to get status of computation EGEE-II INFSO-RI-031688 16

Get result Enabling Grids for E-scienc. E Click to Download Results EGEE-II INFSO-RI-031688 17

Application Specific Portals Enabling Grids for E-scienc. E The Rendering Portal Contact: Mr. Tamas Kiss, kisst@wmin. ac. uk Create a new rendering job by uploading file to be rendered Download rendered images Submit workflow EGEE-II INFSO-RI-031688 18

EMMIL P-GRADE Procurement Portal Enabling Grids for E-scienc. E • Application name: E-marketplace Model Integrated with Logistics • Get result Description of application: – EMMIL facilitates three sided negotiation between buyers, sellers and third party logistics providers aimed at optimising the total costs that has never been offered before. The reason for the lack of integration in the present practice can be the algorithmic and computational complexity involved in the three-sided negotiation mechanism. • Scientific impact of application: – Grid parallelism can help to perform the lengthy computations within a time period that is acceptable for the business environment. The example of EMMIL may encourage researchers to apply more sophisticated algorithms in their business models. • Social impact of application: – Spreading the new model helps to optimise logistics in order to decrease waste of energy and consumption of natural resources and decrease the environmental pollution. • Current user community: – The EMMIL P-GRADE Procurement Portal is used at International Business School for teaching purposes in the frame of “E-business” course (~100 students). EGEE-II INFSO-RI-031688 19

Application Specific Portals (contd. ) Enabling Grids for E-scienc. E Supplier chain portal: EMMIL P-GRADE Procurement Portal Set up the most optimal supplier chain Log in as buyer supplier, or logistic service provider. Create auctions or take your bids Best buyer-seller-logistic service provider pairs discovered, result presented in an Excel file Download rendered images EGEE-II INFSO-RI-031688 Parameter study optimization workflow is automatically generated and executed 20

Enabling Grids for E-scienc. E Part II Support of legacy codes, code reusability and grid interoperability EGEE-II INFSO-RI-031688 21

GEMLCA P-GRADE Portal 2. 4. 1 Enabling Grids for E-scienc. E • • P-GRADE Portal extended with GEMLCA Grid service back-end – To share jobs and legacy codes as application components with others – A step towards collaborative e-Science Developed by the University of Westminster (London) Support for Globus 4 grids (besides GT 2 and EGEE) Available on the UK NGS and OGF GIN Portals • http: //www. cpc. wmin. ac. uk/ngsportal/index. php LCG / g. Lite VOs Legacy code Globus 2 VOs P-GRADE Portal Legacy code job job GEMLCA EGEE-II INFSO-RI-031688 Globus 4 VOs Legacy code 22

Enabling Grids for E-scienc. E Combine legacy codes with new codes inside the same workflow! Combining legacy and non-legacy (job) components Job submission Code invocation Publish successfully executed new code as legacy code EGEE-II INFSO-RI-031688 Job submission 23

NGS P-GRADE Portal Enabling Grids for E-scienc. E • NGS P-GRADE portal is a production service of the UK NGS • It has several advanced extensions to the original P-GRADE portal: – Integration with the GEMLCA legacy code service – Integration with SRB – Integration with OGSA-DAI • The next part of the lecture shows these features of the NGS P-GRADE portal EGEE-II INFSO-RI-031688 24

Enabling Grids for E-scienc. E Grid. FTP, SRM, SRB and OGSA-DAI workflow level integration Grid infrastructure Portal server LOCAL INPUT FILES LOCAL OUTPUT FILES Grid. FTP servers User level storage LOCAL INPUT FILES LOCAL OUTPUT FILES Workflow level Interoperation of local, Grid. FTP, SRM and SRB file catalogues and databases exposed by OGSA-DAI EGEE-II INFSO-RI-031688 REMOTE INPUT FILES REMOTE OUTPUT FILES Computing resources SRB servers Data manipulation Input to workflows Control of remote input/output Output from workflows EGEE Storage elements OGSADAI services 30

Enabling Grids for E-scienc. E Next steps How to get involved www. eu-egee. org EGEE-II INFSO-RI-031688

Portal installations Enabling Grids for E-scienc. E • P-GRADE Portal services: – SEE-GRID infrastructure – Central European VO of EGEE – GILDA: Training VO of EGEE – Many national Grids (UK National Grid Service, Hun. Grid, Turkish Grid, etc. ) – US Open Science Grid, Tera. Grid – OGF Grid Interoperability Now (GIN) VO – … Portal services and account request: http: //portal. p-grade. hu/index. php? m=3&s=0 Account request form on portal login page EGEE-II INFSO-RI-031688 34

Installing P-GRADE Portal Enabling Grids for E-scienc. E – (If you are the administrator of a Grid/VO then) get the installation package from Source. Forge § http: //sourceforge. net/projects/pgportal § Server requirements http: //portal. p-grade. hu/index. php? m=13&s=0 § Local user support must be provided by you – If you are member of an EGEE / Globus VO that does not have P-GRADE Portal yet, then § Ask SZTAKI to add your VO to the Multi-grid P-GRADE Portal Installation § Ask the manager of your VO to install P-GRADE Portal EGEE-II INFSO-RI-031688 35

Support services Enabling Grids for E-scienc. E http: //portal. p-grade. hu • Training: – Training events, trainer community: email list: pgrade-training@lpds. sztaki. hu – Portal to access GILDA resources – Online tutorials • User forum • Installation and operational support team • Application Development and Porting Support: www. lpds. sztaki. hu/gasuc EGEE-II INFSO-RI-031688 37

Enabling Grids for E-scienc. E Learn once, use everywhere Develop once, execute anywhere Thank you! www. portal. p-grade. hu pgportal@lpds. sztaki. hu www. eu-egee. org EGEE-II INFSO-RI-031688