Скачать презентацию DEISA Towards a European HPC Infrastructure www deisa Скачать презентацию DEISA Towards a European HPC Infrastructure www deisa

4dd00e6024841df9eef5404c5fd52a89.ppt

  • Количество слайдов: 25

DEISA Towards a European HPC Infrastructure (www. deisa. eu) Topics Vision The DEISA/e. DEISA DEISA Towards a European HPC Infrastructure (www. deisa. eu) Topics Vision The DEISA/e. DEISA Infrastructure The DEISA 2, the European HPC infrastructure

Status of the European HPC Infrastructure Vision of the European DEISA/e. DEISA projects in Status of the European HPC Infrastructure Vision of the European DEISA/e. DEISA projects in FP 6 To enhance Europe’s capability computing and science by the integration of Europe’s most powerful supercomputing systems Tier-1 in a European HPC e-infrastructure DEISA built a European Supercomputing Service on top of existing national services. This service is based on the deployment and operation of a persistent, production quality, distributed supercomputing environment with continental scope Vision of the European Strategy Forum on Research Infrastructures (ESFRI) Delivering a turnkey operational solution for a future persistent European HPC ecosystem integrating national Tier-1 centres and the new European Tier-0 centres 3/18/2018 2

Vision of a European HPC Ecosystem in FP 7 DEISA 2 - The Infrastructure Vision of a European HPC Ecosystem in FP 7 DEISA 2 - The Infrastructure for the European HPC Ecosystem Deep operational and technological integration of European HPC (T-0 and T-1) centres and systems providing efficient seamless access to shared HPC resources and large data repositories T-0: future shared European Peta. FLOP designing and approving an operational model for a large European Virtual HPC Centre. systems T-1: leading national supercomputing systems Providing scientists access to a large distributed HPC environment via integrated services. DEISA is paving the way to the efficient operation of the T-0 and T-1 ecosystem T-1 T-0 T-1 … T-1 T-0 Enhancing the performance PRACE Building a world-class pan-European High Performance HPC Ecosystem which is operated under the umbrella of an European Legal Entity adopting operational and technological concepts and services designed and approved by DEISA 2. 3/18/2018 3

Towards a European HPC Infrastructure – DEISA 2 USA HPC National Center Tera. Grid, Towards a European HPC Infrastructure – DEISA 2 USA HPC National Center Tera. Grid, Do. E (T 1) Fusion Community National HPC Weather forecast Center Communities (T 1) National HPC Center (T 1) Japan Material NAREGI Sciences Community European Petascale European HPC Center (T 0) Operation - Technology - Application National Data European Petascale DEISA 2 –Center Virtual The (T 0) DEISA 2 National Data Center Russian Climate Academy. National HPC Center Community Objectives (T 1) European Petascale Center (T 0) Center (T 1) Chinese Industry Academy Communities National HPC Center (T 1) Cosmology Community Enhancing the existing distributed European HPC environment (DEISA) to a turnkey operational infrastructure Advancing the computational sciences in Europe by supporting user communities and extreme computing projects Enhancing the service provision by offering a complete variety of options of interaction with computational resources Integration of T 1 and T 0 centres The Petascale Systems need a transparent access from and into the national data repositories Bridging worldwide HPC projects 3/18/2018 4

DEISA Partners DEISA: May 1 st, 2004 – April 30 th, 2008 Three new DEISA Partners DEISA: May 1 st, 2004 – April 30 th, 2008 Three new partners joined June 2005 (e. DEISA) DEISA 2: May 1 st, 2008 – April 30 th, 2011 3/18/2018 5

Partners/ Associate Partners BSC Barcelona Supercomputing Centre Spain CINECA Consortio Interuniversitario per il Calcolo Partners/ Associate Partners BSC Barcelona Supercomputing Centre Spain CINECA Consortio Interuniversitario per il Calcolo Automatico Italy CSC Finnish Information Technology Centre for Science Finland EPCC University of Edinburgh and CCLRC ECMWF European Centre for Medium-Range Weather Forecast FZJ Research Centre Juelich HLRS High Performance Computing Centre Stuttgart Germany IDRIS Institut du Développement et des Ressources France UK UK (int) Germany en Informatique Scientifique - CNRS LRZ Leibniz Rechenzentrum Munich Germany RZG Rechenzentrum Garching of the Max Planck Society Germany SARA Dutch National High Performance Computing KTH Kungliga Tekniska Högskolan CSCS Swiss National Supercomputing Centre Switzerland JSCC Joint Supercomputer Center of the Russian Russia Netherlands Sweden Academy of Sciences 3/18/2018 6

The basic DEISA infrastructures and services Dedicated high speed network infrastructure Common AAA infrastructure The basic DEISA infrastructures and services Dedicated high speed network infrastructure Common AAA infrastructure Global data management infrastructure Integrating distributed data with distributed computing platforms, including hierarchical storage management and databases. Major highlights are: High performace remote I/O and data sharing with global file systems, using full network bandwidth High performance transfers of large data sets, using full network bandwidth DCPE (Distributed Common Production Environment) The job management service The science gateways (portals) to supercomputing resources Common Operational Environment Common monitoring and Information systems Common system operation Common help desk Global Application Support 3/18/2018 7

DEISA network infrastructure SURFnet FUNET DFN UKERNA GÉANT GARR RENATER Red. Iris 1 Gb/s DEISA network infrastructure SURFnet FUNET DFN UKERNA GÉANT GARR RENATER Red. Iris 1 Gb/s GRE tunnel 10 Gb/s wavelength 10 Gb/s routed 10 Gb/s switched 3/18/2018 8

GARR CINECA-Net: 130. 186. 27. 0/24 DEISA 1 GE site ECMWF via GEANT 2 GARR CINECA-Net: 130. 186. 27. 0/24 DEISA 1 GE site ECMWF via GEANT 2 GRE tunnel ECMWF-Net: 136. 156. 48. 0/24 3/18/2018 9

DEISA Data Services - Global File System (based on IBM’s GPFS) AIX IBM domain DEISA Data Services - Global File System (based on IBM’s GPFS) AIX IBM domain ECMWF (UK) Linux SGI IDRIS (FR) RZG (DE) HPC Common Global File System CSC (FI) SARA (NL) High Peformance Common Global File System similar architectures / operation systems various architectures / operating systems High bandwidth (10 Gbit/s) High bandwidth (up to 10 Gbit/s) LRZ (DE) BSC (ES) CINECA (IT) FZJ (DE) LINUX Power-PC 3/18/2018 10

Workflow simulations using UNICORE supports complex simulations that are pipelined over several heterogeneous platforms Workflow simulations using UNICORE supports complex simulations that are pipelined over several heterogeneous platforms (workflows). UNICORE handles workflows as a unique job and transparently moves the output – input data along the pipeline. UNICORE clients that monitor the application can run in laptops. UNICORE has a user friendly graphical interface. DEISA has developed a command line interface for UNICORE. 3/18/2018 11

Operational Model 3/18/2018 12 Operational Model 3/18/2018 12

Enabling Science The basic service providing model for scientific users is the Extreme Computing Enabling Science The basic service providing model for scientific users is the Extreme Computing Initiative (see www. deisa. org) Identification, deployment and operation of a number of « flagship » applications requiring the infrastructure services, in selected areas of science and technology. European Call for proposals in May-June every year. Applications are selected on the basis of scientific excellence, innovation potential and relevance criteria, with the collaboration of the HPC national evaluation committees. Supported by the Applications Task Force (ATASKF) Hyperscaling of huge parallel applications, data oriented applications Workflows and coupled applications Production of an European Benchmark Suite for HPC systems 3/18/2018 13

DEISA Extreme Computing Initiative DECI call 2005 51 proposals, 12 European countries involved 30 DEISA Extreme Computing Initiative DECI call 2005 51 proposals, 12 European countries involved 30 mio cpu-h requested 29 proposals accepted, 12 mio cpu-h (standardized of P 4 Jülich) DECI call 2006 41 proposals, 12 European countries involved 28 mio cpu-h requested 23 proposals accepted, 12 mio cpu-h DECI call 2007 63 proposals, 14 European countries involved (US, Canada, Brazil, Israel) 30 mio cpu-h requested 3/18/2018 14

DEISA Extreme Computing Initiative 15 European countries so far involved in DECI: Austria Greece DEISA Extreme Computing Initiative 15 European countries so far involved in DECI: Austria Greece Poland Sweden Finland Hungary Portugal Switzerland France Italy Russia UK 3/18/2018 Germany Netherlands Spain 15

DECI Projects – Publications Cover story of Nature May 24, 2007 DECI project POLYRES DECI Projects – Publications Cover story of Nature May 24, 2007 DECI project POLYRES Benedict J. Reynwar, Gregoria Illya, Vagelis A. Harmandaris, Martin M. Müller, Kurt Kremer & Markus Deserno: Aggregation and vesiculation of membrane proteins by curvature mediated interactions, NATURE Vol 447|24 May 2007| doi: 10. 1038/nature 05840 26 publications so far, among them NATURE, Phys Rev B, Phys Rev Lett, PNAS 3/18/2018 16

The purpose of the DEISA 2 Consolidation of the existing DEISA infrastructure Guaranteeing the The purpose of the DEISA 2 Consolidation of the existing DEISA infrastructure Guaranteeing the continuity of those activities and services that currently contribute to the effective support of world-leading computational science in Europe DEISA 2 providing a lean and reliable turnkey operational solution for a persistent European HPC ecosystem Evolvement this European infrastructure towards a robust and persistent European HPC ecosystem Enhancing the existing services, by deploying new services including support for European Virtual Communities, and by cooperating and collaborating with new European initiatives, especially PRACE that will enable shared European Peta. Flop/s supercomputer systems DEISA 2 as the vector for the integration of Tier-0 and Tier-1 systems in Europe 3/18/2018 17

The purpose of DEISA 2 advancing computational sciences in Europe DEISA 2 will provide The purpose of DEISA 2 advancing computational sciences in Europe DEISA 2 will provide a consistent infrastructure for virtual organisations as the European Fusion Development Agreement (EFDA) supporting ITER, the Climate Modelling Community, the Cosmology Consortium VIRGO, Material Sciences projects/communities, Life Science projects/communities DEISA 2 as an extended partnership DEISA 2 targets the integration of further national Tier-1 partners DEISA 2 bridging worldwide HPC projects Interoperation in a Grid context worldwide is seen as an important step for supporting international science communities which acquire computing time traversing existing political boundaries 3/18/2018 18

Integrated Tier-1 Systems Participant short name MPG-RZG Architecture IBM Blue. Gene P TF/s[1] #Cores[2] Integrated Tier-1 Systems Participant short name MPG-RZG Architecture IBM Blue. Gene P TF/s[1] #Cores[2] Memory in TB[3] 27. 0 4. 0 120. 0 IBM Power 6 8, 192 6, 400 20. 0 BAd. W-LRZ SGI-Altix Itanium 62. 3 9, 728 39. 0 BSC IBM Power. PC 94. 2 10240 20. 0 CINECA IBM Blade Center 26. 6 5120 10. 0 IBM Power 5 3. 9 512 1. 0 Cray XT 4 dc 10. 1 2, 024 2. 0 Cray XT 4 qc 70. 0 6, 736 7. 8 223. 0 65, 536 32. 0 8. 9 1, 312 5. 0 > 50 n/a 6. 7 1800 10 IBM p 690+ successor > 40 n/a IBM Power 5+ 14. 6 1920 7. 7 IBM Power 6 60. 2 3328 15. 6 UEdin-EPCC Cray XT 4 65. 0 11, 520 35. 0 UStutt-HLRS NEC SX-8 13. 0 576 9. 0 CSC FZJ IBM Blue. Gene P IBM p 690 cluster JUMP successor IDRIS-CNRS SARA IBM p 690+ Cluster 3/18/2018 19

Collaboration with other EU Projects, Institutions, … Site, Project, Community Short Description CLARIN The Collaboration with other EU Projects, Institutions, … Site, Project, Community Short Description CLARIN The pan-European Common Language Resources and Technology Infrastructure is committed to enabling e. Humanities. COSMOS The COSMOS Network gathers 20 institutions from 6 different European countries involved in the climate research. DANTE (Delivery of Advanced Network Technology to Europe) is a non-profit organisation owned by a group of National Research and Education Networks (NRENs). EFDA/ITER The European Fusion Development Agreement is interested in a strong support for theoretical fusion science community. EGI The consortium of EGI aims at establishing a sustainable Grid infrastructure in Europe, coordinating national Grid initiatives. ENES The European Network for Earth System Modelling expects a strong support for the earth science virtual community in a pan-European context by DEISA 2. European Psi-k Network in Material Science Psi-k Network will collaborate with DEISA 2 in improving the support for the European Virtual Scientific Communities in the field of Material Science. Euro. Physiome A European biomedical research project. HPC-Europa Pan-European Research Infrastructure on HPC providing HPC access and scientific support to researchers. ITESM The computing department of the Instituto Tecnológico y de Estudios Superiores de Monterrey, Campus Estado de provides a wide range of Grid infrastructures for Latin America. Joint Supercomputer Center of the Russian Academy of Sciences The Joint Supercomputer Center JSCC represents HPC in the Russian Academy of Sciences. They are considered an Associate Partner. Meraka The African Advanced Institute for Information and Communications Technology is the leading HPC centre in South Africa and is deeply involved in the South African e-infrastructure. NAREGI The National Research Grid Initiative is the leading Japanese HPC infrastructure project. OMII-Europe is the interoperability project in Europe providing open standards based interoperability components. PACE Partnership for Advanced Computing in Europe is aiming at installing leadership class supercomputers in Europe. SSC The Shanghai Supercomputer Center is currently the most powerful HPC centre in China. Supercomputing Center of the Chinese Academy of Sciences The Supercomputing Center SCCAS represents HPC in the Chinese Academy of Sciences. Tera. Grid is an open scientific discovery infrastructure in the US, combining leadership class resources at nine partner sites to create an integrated, persistent computational resource. 3/18/2018 20

DEISA Towards a European HPC Infrastructure Topics Vision The DEISA/e. DEISA Infrastructure The DEISA DEISA Towards a European HPC Infrastructure Topics Vision The DEISA/e. DEISA Infrastructure The DEISA 2, the European HPC infrastructure

Relationship of the WPs in DEISA 2 (except management, dissemination and one “real” JRA) Relationship of the WPs in DEISA 2 (except management, dissemination and one “real” JRA) WP 3 - Operations Request for new/upgraded services Answer Offer new service or upgrade existing ones WP 4 - Technologies Request for new/upgraded services Answer Offer new service or upgrade existing ones WP 6/WP 7/WP 9 – Applications and user 3/18/2018 support 22

Management Structure 3/18/2018 23 Management Structure 3/18/2018 23

DEISA 2 WP 1 Management WP 2 Dissemination, External Relations WP 3 Operations WP DEISA 2 WP 1 Management WP 2 Dissemination, External Relations WP 3 Operations WP 4 Technologies WP 5 Applications enabling WP 6 Environment, User Related Application Services, and Training WP 7 DECI Computing Projects and Benchmark Suite WP 8 (JRA) Integrated DEISA Development Environment WP 9 (JRA) Enhancing Scalability 3/18/2018 24

DEISA Partners DEISA Start on 1 st of May 2004 Three new partners joined DEISA Partners DEISA Start on 1 st of May 2004 Three new partners joined with e. DEISA June 2005 DEISA 2 should start 1 st of May 2008 with 3 new Associate Partners for 3 years 3/18/2018 25