Enabling Grids for E-scienc. E Conjugating Science Gateways and Grid Portals into e-Collaboration environments: the Liferay and GENIUS/Engin. Frame use case Riccardo Rotondo (riccardo. rotondo@ct. infn. it) INFN Catania - Italy Tera. Grid ‘ 10 Pittsburgh, 2 -5 August 2010
Outline Enabling Grids for E-scienc. E • • Who’s INFN ? Who’s NICE ? The e-Science paradigm and its adoption “hurdles”; The current GENIUS portal: new functionalities and current limitations; • The approach towards a new e-collaboration environment: – The new GENIUS architecture; – Liferay; – Engin. Frame 2010; • Demo; • Future work; • Summary and conclusions. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 2
The INFN (www. infn. it) Enabling Grids for E-scienc. E • The INFN - the Italian National Institute of Nuclear Physics – is an organization dedicated to the study of the fundamental constituents of matter, and conducts theoretical and experimental research in the fields of sub-nuclear, and astro-particle physics. Fundamental research in these areas requires the use of cutting-edge technologies and instrumentation, which the INFN develops both in its own laboratories and in collaboration with the world of industry. These activities are conducted by INFN in close collaboration with the academic world. PARTICLE PHYSICS ASTROPARTICLE PHYSICS NUCLEAR PHYSICS THEORETICAL PHYSICS TECHNOLOGY RESEARCH • INFN is organized in 19 Divisions, located at university physics departments, and 4 National Laboratories located in Catania, Legnaro, Rome and under the Gran Sasso mountain; • The INFN workforce includes about 2000 of its own employees, almost 2000 university employees involved in research conducted by the Institute, and 1300 young researchers, including undergraduate and graduate students and research fellows. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 3
The INFN Grid Project Enabling Grids for E-scienc. E (grid. infn. it) • The INFN Grid project is an internal special initiative used by INFN to develop and deploy the Grid middleware services which allow its various user communities, while taking part in worldwide experiments and projects, to transparently and securely share the computing and storage resources together with the applications and technical facilities available in the different administrative domains of the various Institutions and geographical sites. – More information at http: //grid. infn. it/modules/IG_history/. • INFN operates since 2001 a country-wide production quality Grid Infrastructure made, as of today, of almost 40 sites located all over Italy; • The INFN Grid Infrastructure counts more than 16, 000 CPU cores and several Petabytes of storage, both on disks and tapes; • The INFN Grid Infrastructure provide computing and storage services not only to the High Energy Physics community but also to several other sciences such as Astrophysics, Bioinformatics, Biomedicine, Chemistry, Earth Science, etc. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 4
Enabling Grids for E-scienc. E EU Projects tagged with INFN in the “Grid World” are those where INFN manages coordination tasks. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 5
About NICE Enabling Grids for E-scienc. E 3 Founded 1996, profitable, self-funded NICE as ISV (Independent Software Vendor) – Mission: facilitate access to Technical and Cloud Computing applications, data and resources – Engin. Frame product line for Technical Computing Portal (web and web-services) – – Cloud. Frame solution for Cloud Platform-as-a-Service Global partnerships in HPC and Visualization NICE as solution provider – 14 years experience with Enterprise Technical Computing throughout all industries – Vertical and horizontal solutions – Cloud Computing solutions
~100 Customers. . . and growing Enabling Grids for E-scienc. E Energy & Utilities Aerospace & Manufacturing Addax Petroleum, AECL, BG, Chevron, Conoco. Phillips, DSC-Libya, ENI, Exxon. Mobil, Hess, Marathon Oil, Nexen, Rosneft, Schlumberger, Senergy, Sibneft, Sinopec, Slavneft, Sonatrach, Statoil, Talisman Energy, TNK-BP, TNNC, TOTAL, Tyumen. NIIGaz, VNIIGaz, Western. Geco, AIRBUS, Air Products and Chemicals, Procter&Gamble, Galileo Avionica, Hamilton Sunstrand, Kimberly Clark, Magellan Aerospace, MTU, Pratt & Whitney, Raytheon, Simpson Strong-Tie Life Sciences Bayer, Lit. Bio project, DEISA project, Biolab, Swiss Institute for Bioinformatics, Partners Healthcare, Pharsight, M. D. Anderson Cancer Center Audi, ARRK, Bridgestone, Bosch, Corus Automotive, Delphi, Elasis/CRF, Ferrari, Brawn GP, Jaguar-Land. Rover, Magneti Marelli, Mc. Laren, P+Z, PSA, Red. Bull Engineering, Swagelok, Suzuki, Toyota, TRW, Volkswagen High Tech Research & Education STMicroelectronics, Accent, Samsung SDI, Sensor. Dynamics, Motorola CCLRC, CERN, CILEA, CINECA, CNR, CNRS/IN 2 P 3, ENEA, Georgia State Uni, ICI, , INFN, ITEP, Harvard Business School, RMSC, SSC-Russia, SDSC, Ferrara Uni, ITU, T. U. Dresden, Trinity College Dublin, Huazhong Normal Uni, Yale University Financial Services Uni. Credit, Sal Oppenheim Automotive & Industrial Equipment
Computationally intensive research Enabling Grids for E-scienc. E Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 8
Computationally intensive research Enabling Grids for E-scienc. E Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 9
Enabling Grids for E-scienc. E e-Science Virtual Organisations e-Infrastructure Applications Data Instruments Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 10
Enabling Grids for E-scienc. E e-Infrastructure at a “world” scale! Tier-1 Centers: TRIUMF (Canada); Grid. KA(Germany); IN 2 P 3 (France); CNAF (Italy); SARA/NIKHEF (NL); Nordic Data Grid Facility (NDGF); ASCC (Taipei); RAL (UK); BNL (US); FNAL (US); PIC (Spain) Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 11
Enabling Grids for E-scienc. E MPIType = "MVAPICH 2_PGI 706"; Cpu. Number = 16; MPIGranularity = 4; JDL Executable = "flash 2"; Std. Output = "mpi. out"; Std. Error = "mpi. err"; Input. Sandbox = {"watchdog. sh", "mpi. pre. sh", "mpi. post. sh", "flash. par", "summers_den_ 1 e 0. rates", "flash 2"}; Output. Sandbox = {"mpi. err", "mpi. out", "watchdog. out", "flash_bubble. log", "amr_log"}; Requirements = (other. Glue. CEUnique. Id == "unipa-ce(other. Glue. CEUnique. Id 01. pa. pi 2 s 2. it: 2119/jobmanager-lcglsf-hpc"); Retry. Count = 0; echo Staging Input Data (Courtesy of European Space Agency); #edg-rm --vo=gilda copy. File lfn: $1. N 1 file: //$PWD/$1. N 1; lcg-cp --vo=gilda lfn: $1. N 1 file: //$PWD/$1. N 1; echo Staging Application; gunzip beam 20. tar. gz; tar xvf beam 20. tar; cd beam-2. 0/bin; echo Starting Application; . /pds 2 jpg-ASAR-run. sh $1; mv $1 -b*. jpg. . /. . cd. . /. . rm -fr beam-2. 0; rm -fr $PWD/$1. N 1; rm -fr $PWD/beam 20. tar; echo Input ENVISAT Product courtesy of European Space Agency touch ENVISAT_Product_courtesy_of_European_Space_Agency echo No Output Packaging; echo Done!; Scripts Type = "Job"; Job. Type = "MPICH"; But…using Grids is not straightforward $ voms-proxy-info --all subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=Mr. Bayes/CN=proxy issuer : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=Mr. Bayes/CN=proxy identity : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=Mr. Bayes/CN=proxy type : proxy strength : 1024 bits path : /tmp/x 509 up_u 512 timeleft : 500: 38: 44 === VO gilda extension information === VO : gilda subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=Mr. Bayes issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms. ct. infn. it attribute : /gilda/Role=NULL/Capability=NULL attribute : /gilda/generic-users/Role=NULL/Capability=NULL attribute : /gilda/idl/Role=NULL/Capability=NULL timeleft : 23: 59: 33 uri : voms. ct. infn. it: 15001 CLI $ glite-wms-job-status https: //gilda-lb-01. ct. infn. it: 9000/1 o 4 BVjqg 2 t. J 4 r. NXTAIGAg ******************************* BOOKKEEPING INFORMATION: Status info for the Job : https: //gilda-lb-01. ct. infn. it: 9000/1 o 4 BVjqg 2 t. J 4 r. NXTAIGAg Current Status: Done (Success) Logged Reason(s): - Job terminated successfully Exit code: 0 Status Reason: Job terminated successfully Destination: grid 010. ct. infn. it: 2119/jobmanager-lcgpbs-gilda Submitted: Tue Jun 29 15: 34: 40 2010 CEST ******************************* GSI
A viable solution ? The GENIUS Grid Portal (INFN-NICE collaboration – https: //genius. ct. infn. it) Enabling Grids for E-scienc. E “robot” certificates on “e-tokens”
Robot certificates in a nutshell Enabling Grids for E-scienc. E • Robot certificates have been introduced by several Certification Authorities to allow users, who are not familiar with personal certificates and do not belong to any Vos, to access and use Grids. – They are extremely useful, for instance, to automatize grid service monitoring, data processing production, distributed data collection systems, etc. ; – Basically, these certificates can be used to identify a person responsible for an “unattended” service or process acting as client and/or server. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 14
Robot certificates & e. Tokens Enabling Grids for E-scienc. E • In order to strong reduce the risks to have the portal certificate compromised, the INFN CA has decided to issue these special certificates on board of Aladdin e. Token PRO 32 K smart cards. • Each smart card can support several (up to 32) robot certificates: one for each application available on the grid portal. • GENIUS fully supports robot certificates ! Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 15
GENIUS and the robot certificates – The User Tracking tool Enabling Grids for E-scienc. E proxy create a obot 2. r with the cate certifi 1. ask for a service User 5. get the results action 4. get output ck. tra 2’, 3’ ser u Admin 3. exec ute The Grid 6/7. query L&B for accounting data L&B Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 16
User Tracking System (1/2) Enabling Grids for E-scienc. E Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 17
User Tracking System (2/2) Enabling Grids for E-scienc. E Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 18
Current GENIUS pros and cons Enabling Grids for E-scienc. E • Pros: – Hides completely the complexity of the underlying middleware; – Shows the same GUI to the end users as the middleware changes/evolves; – Through the support to robot certificates, allows everybody to access and use the Grid greatly reducing the slope of the learning curve for non expert users; – Can be easily customized to expose services specific to a particular VO; • Cons: – Does not include Web 2. 0 functionalities; – It is quite monolithic and it is very difficult to integrate it inside thematic portals (e. g. , science gateways). Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 19
The new GENIUS architecture Enabling Grids for E-scienc. E Single Sign-On 2010 Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 20
Liferay (www. liferay. com) Enabling Grids for E-scienc. E • Highly-configurable, scalable, open source portal framework; • Compatible with JSR 168/268 standards and based on modern web 2. 0 technologies; • Liferay services planned to be used: – Portal; – CMS & WCM; – Collaboration and “social” software. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 21
Engin. Frame 2010 architecture Enabling Grids for E-scienc. E Services Gateway Portal Applications Data Security Monitoring & Management Distributed Resource Manager (LSF, PBS, MOAB, Torque/MAUI, SGE, Win. HPC, i. RODS, g. Lite) Remote 2 D/3 D Viz. (tight. VNC) Virtual/Physical Provisioning Distributed and Heterogeneous Computational Resources Distributed Data Management Distributed and Heterogeneous Data Resources
Engin. Frame 2010 new GUI (1/3) (based on the latest Web 2. 0 and AJAX technologies) Enabling Grids for E-scienc. E Very unobtrusive and easy to use Modern, elegant and visually appealing Designed to help organizations and people to work more efficiently and to get more done in less time Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 23
Engin. Frame 2010 new GUI (2/3) (based on the latest Web 2. 0 and AJAX technologies) Enabling Grids for E-scienc. E • • • Sorting Filtering Searching Pagination Charts Commands Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 24
Engin. Frame 2010 new GUI (3/3) (based on the latest Web 2. 0 and AJAX technologies) Enabling Grids for E-scienc. E The RFB (Remote File Browser) dialog has been completely renewed to make server-side (and Grid) input data selection even easier The new file manager component allows to seamless navigate and access server-side (and Grid) files from the web browser
What has been done so far …and what’s missing Enabling Grids for E-scienc. E • Done: – Single Sign-On across Liferay, Engin. Frame and the Grid with robot certificate and VOMS proxy extensions; – Co-existence of Liferay and Engin. Frame on the same physical machine but with two different Tomcat instances; – Engin. Frame services rendered as Liferay portlets; • To do: – Evaluate the need to make Liferay and Engin. Frame co-exist within the same Tomcat instance; – Improvement of the look&feel (so, do not expect beautiful web pages during the demo ). Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 26
Demo: Large scale phylogenetic analyses with Mr. Bayes Enabling Grids for E-scienc. E WNs WNs SSO http: //www. youtube. com/watch? v=oo 5 F 0 N 9 H 9 EQ Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 27
DEMO Enabling Grids for E-scienc. E Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 28
Enabling Grids for E-scienc. E Future work 1. 2010 3. 1. Hooks to ecollaboration and e -learning tools 2. Engin. Frame services fully exposed as Liferay portlets 3. Creation of generic APIs to call different middleware Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 29
Summary and conclusions Enabling Grids for E-scienc. E • After several years of experience with GENIUS and Engin. Frame, the work presented here will break with the past; • We won’t provide any more top-down solutions; we will instead work with NICE to create “bricks” that could be used to quickly build thematic science gateways; • Engin. Frame is a commercial product fully based on open standards and open platforms; not-for-profit projects with educational and research purposes will be granted with free licenses; first line support could be provided by INFN or, if requested, by NICE; • We will test this new approach in the framework of two European Grid projects: DECIDE (life sciences) and INDICATE (cultural heritage) that are going to start on the 1 st of September 2010; • If there is any interest in the Tera. Grid community, we are of course, open to discuss synergies and possible collaborations, especially on the extension of the Grid Layer to US middleware. Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 30
Acknowledgments Enabling Grids for E-scienc. E • • Roberto Barbera 1, 2 (roberto. barbera@ct. infn. it) Giuseppe La Rocca 2 (giuseppe. larocca@ct. infn. it) Alberto Falzone 3 (alberto. falzone@nice-software. com) Paolo Maggi 3 (paolo. maggi@nice-software. com) Nicola Venuti 3 (nicola. venuti@nice-software. com) Luca Carrogu 3 (luca. carrogu@nice-software. com) Enrico Usai 3 (enrico. usai@nice-software. com) 1. Department of Physics and Astronomy of the University of Catania - Italy 2. Italian National Institute of Nuclear Physics, Division of Catania - Italy 3. NICE srl - Italy Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 31
Important announcement Enabling Grids for E-scienc. E Submission deadline: August 31, 2010 Workshop dates: September 20 -22, 2010 • http: //agenda. ct. infn. it/event/iwsg 2010 Teragrid ‘ 10 Pittsburgh PA – August 5 2010 - 32
Скачать презентацию Enabling Grids for E-scienc E Conjugating Science Gateways
1bfd2d782b52b42dfbd83fa82530719a.ppt