EGI-In SPIRE Shared Services and Tools MPI John

EGI-In. SPIRE Shared Services and Tools MPI John Walsh, Isabel Campos, Antonio Laganà EGITF-2010, Amsterdam EGI-In. SPIRE RI-261323 www. egi. eu

MPI in EGI • MPI support in ARC, g. Lite, UNICORE • JS mechanisms in M/Ws differ radically – ARC: x. RSL + RTE – g. Lite: JDL with MPI-START wrappers – UNICORE: JSDL • HUCs will exploit MPI enabled resources made available through these middlewares EGI-In. SPIRE RI-261323 www. egi. eu

MPI Heavy Users • Comp. Chem & Fusion – Very early drivers of MPI on EGEE • But also other major scientific discipline! – Astronomy & Astrophysics – Biomed and Life Sciences – Earth Sciences – …… EGI-In. SPIRE RI-261323 www. egi. eu

MPI in EGI-In. SPIRE • Heavy User Community focus – Task TSA 3. 2. 5 (Support for HUC) – 54 PM over three years (CSIC, TCD, UNIPG) • SA 1 Early Adopters • SA 1 effort from NGIs to deal with acc/ issue • MPI related components from EMI – MPI-START and glite-MPI_utils, YAIM MPI – Alternative/Custom MPI packaging (optional) • Support better batch system integration EGI-In. SPIRE RI-261323 www. egi. eu

People • • • Isabel Campos (CSIC) Antonio Laganà (UNIPG) Alessandro Costantini (UNIPG) * Enol Fernández del Castillo (CSIC) Stefano Crocchianti (UNIPG) Osvaldo Gervasi (UNIPG) Carlo Manuali (UNIPG) Leonardo Pacifici (UNIPG) * Jamie Shiers (CERN) John Walsh (TCD) EGI-In. SPIRE RI-261323 www. egi. eu

MPI HUC objectives • Improved end-user documentation – Deployment – job submission in ARC, g. Lite and UNICORE – MPI application development • Outreach and dissemination • User community, NGI, site engagement – Gathering direct input and requirements • Standardisation bodies participation • Generic Parallel Job support (GPU/Open. MP) EGI-In. SPIRE RI-261323 www. egi. eu

MPI Task Outputs • MPI Workbenches – Increasing complexity • Simple -> complex workflows – Focus on HUC communities – Reuse by other communities • User and Site Admin surveys – To specify future m/w requirements • Dissemination • MPI Cookbook EGI-In. SPIRE RI-261323 www. egi. eu

Parallel Applications (CCMST) • Basic Linear algebra algorithms (low level complexity codes) – Cannon, Fox & Strassen algorithms • Medium complexity (in house codes) – RWAVEPR quantum time-dependent reactive dynamics – SC-IVR semiclassical -initial value representation- • High complexity (general purpose computational codes) – GAMESS-US high-level ab initio molecular quantum chemistry – DL_POLY molecular dynamics simulation of complex systems – NAMD molecular dynamics simulation of complex systems EGI-In. SPIRE RI-261323 www. egi. eu

MPI in Fusion • Large variety of Applications: Serial, parallel, shared memory, … • BUT: Outstanding Parallel long simualtions (MPI on HPC). GYROKINETIC SIMUATION OF PLASMA TURBULENCE • Example: EUTERPE code (30 days CPU for 300 micro-second evolution on Mare Nostrum at BSC, Spain) • Target: 10 seconds of evolution. EGI-In. SPIRE RI-261323 www. egi. eu

Fusion MPI Workflows KEPLER: Workflow engine Gateway HPC Code (MPI) Grid Code (serial or MPI) Fusion VO EGI-In. SPIRE RI-261323 • Workflows involving Grid-HPCs, serial and parallel applications. • Kepler launches the different actors and organizes the workflow. • Kepler runs on the fusion Gateway for managing the data. • One actor running on HPC: MPI. • And the other on the grid (thousands of jobs): Serial or MPI. Fusion slides courtesy of CIEMAT: Francisco M. Castejon Magaña, Antonio Gomez Iglesias. www. egi. eu

EGI Virtuous Cycle Users Infrastructure Assessment EGI-In. SPIRE RI-261323 Requirements Technology www. egi. eu

Issues • Infrastructure in transition – Integration and interoperability path • Nagios or equivalent for MPI testing? • Standards for MPI/Parallel apps/libs? • MPI from some OS distros not suitable • Accounting – Torque deficiencies, potentially others – GPU usage accounting? – Multi-core/Multi-thread? EGI-In. SPIRE RI-261323 www. egi. eu

Questions? EGI-In. SPIRE RI-261323 www. egi. eu