UNICORE and Grid Computing in Europe Dietmar Erwin

UNICORE and Grid Computing in Europe Dietmar Erwin Forschungszentrum Jülich D. Erwin@fz-juelich. de August 27, 2003 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG

Contents • • Introduction to FZJ Objectives and Status of UNICORE Grid Projects in Europe Eurogrid, GRIP and OGSA Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 2

“The Future is Our Mission” Matter Energy Information Life Environment Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG Source FZJ 3

Forschungszentrum Jülich at a Glance Founded: Dec. 1956 Area: 2. 2 km 2 Operator: • Federal Republic of Germany (90 %) • State of North Rhine-Westphalia (10 %) Budget: Bodies: • Partners Meeting • Supervisory Board • Board of Directors • Scientific and Technical Council Staff: Legal Form: PLC Institutional: € 236 Mio/year Non-institutional: € 52 Mio /year Scientific/Technical Structure: • 10 Departments (39 Institutes) • 7 Central Departments • 2 Programme Groups • 3 Project Management Groups 4200 Including: • 1000 Scientists • 400 Postgraduate Students • 150 Undergraduate Students • 300 Trainees Visiting Scientists: more than 700 annually from more than 30 countries Forschungszentrum Jülich HPN Workshop via AG in der Helmholtz-Gemeinschaft Source FZJ 4

Objectives UNICORE: Uniform Interface to Computing Resources Developed software for a • seamless, • secure, and • intuitive access to distributed computing resources http: //www. unicore. org Funded by BMBF, grant: 01 IR 001 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 5

UNICORE Highlights • Intuitive, system-independent creation and control of jobs • Support for multi-system and multi-site jobs • Dynamic flow control • Integrated security through X. 509 certificates • Access to remote file stores and archives • Extensible support for scientific & commercial applications • Minimal intrusion into site autonomy Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 6

UNICORE Screenshot Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 7

UNICORE Architecture List of Sites User Certificate Preparation and Control of jobs UNICORE Client Job Preparation Agent (JPA) Job Monitor Controller (JMC) AJO Unsafe Internet (SSL/https) Abstract status request optional firewall User authentication Site-specific authorization User mapping, Resources info Network Job Superviser (NJS) Job incarnation IDB Incarnated job in der Helmholtz-Gemeinschaft optional firewall Sending jobs to other gateways, data transfer NJS IDB UUDB . . . UUDB IDB Status request files Batch Subsystem Forschungszentrum Jülich LRZ UNICORE Gateway Safe Intranet (TCP/http) Target System Interface (TSI) Commands FZJ UNICORE Gateway TSI . . . TSI SV 1 T 3 E Batch Subsystem HPN Workshop via AG SR 8000 Batch Subsystem 8

CPMD Application support Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 9

Ad hoc weather prediction Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG Source DWD C: EUROGRID Projekt 10

UNICORE Status • Project UNICORE Plus successfully completed Report at: http: //www. unicore. org • UNICORE basis for a German HPC Grid (LRZ, HLRS, FZJ) • Software available as Open Source for R&D projects http: //www. unicore. org/download • UNICORE commercially supported and marketed by Pallas www. pallas. com • UNICORE used in European Grid projects • UNICORE selected by Japanese NAREGI project Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 11

UNICORE Deployment LRZ: SR 8000, VPP, LC* RUS: SX-5, 2 x. LC* FZJ: 2 x. T 3 E, SV 1, LC* ZIB: T 3 E, Sun DWD: IBM SP, SGI RUKA: IBM SP PC²: LC* TUD: T 3 E, SGI Pallas: Sun Fujitsu: VPP, Sun ZIB PC 2 FZJ TUD DWD RUKA RUS LRZ *LC = Linux Cluster Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 12

European Projects Applications AVO Euro. Grid GRIA Middleware DAMIEN Mammo. Grid EGSO Se. Le. Ne Bio. Grid Open. Mol. Grid Flow. Grid MOSES GEMSS GRASP GRACE Cross. Grid COG Grid. Lab GRIP Data. Grid Data. TAG Infrastructure Wave 1 Projects Wave 2 Projects Oct 2000 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft Oct 2001 HPN Workshop via AG Source EC Oct 2002 13

FP 6 Grid Funding IST-FP 6 commitment to Grid research • First actions launched in IST-FP 5 • Grid research is a strategic objective in FP 6 125 M€ From FP 5 to FP 6 Grid research funding more than doubles 58 M€ FP 6 FP 5 2000 -2002 -2006 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG Source EC 14

1 Forschungszentrum Jülich, D 2 Pallas Gmb. H, D 3 Fujitsu European Centre for Information Technology, UK 4 debis Systemhaus, D 5 5 Parallab - University of Bergen, N 6 CNRS - IDRIS, F 8 7 Warsaw University - ICM, P 3 7 8 1 Victoria University of Manchester, UK 2 9 4 6 9 Deutscher Wetterdienst, D 11 10 GIE EADS CCR, F 11 Forschungszentrum Jülich ETH Zürich, CH in der Helmholtz-Gemeinschaft 10 HPN Workshop via AG 15

Objectives • Establish a European GRID network of leading European HPC centres • Enhance UNICORE through: – Resource broker – Standard based File Transfer (grid. FTP) • Demonstrate Grid solution in – – Bio molecular simulations, Weather prediction, Coupled CAE simulations, Structural analysis. Funded by EU, grant: IST-1999 -20247 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 16

Grid Architecture UNICORE Application Layer Applications, Portals, Environments Collective Layer Higher Services, APIs, Protocols Resource Layer APIs, Protocols, SDKs Fabric Layer Forschungszentrum Jülich in der Helmholtz-Gemeinschaft Globus Computers, Networks, Devices HPN Workshop via AG 17

GRIP Objectives 1. To develop interoperability software between UNICORE and Globus based Grid systems 2. To develop and demonstrate techniques for inter-Grid applications in meteorology and bio-molecular science 3. To influence creation of international Grid standards through the Global Grid Forum (GGF) GRIP partners - lead GGF working/ research groups - work on the definition of OGSA, the Open Grid Service Architecture Seamless, secure, intuitive access to different systems and sites to solve complex problems Grid Interoperability: Creation of virtual organizations using different technologies based on common standards Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 18

GRIP Partners Forschungszentrum Jülich (FZJ) Pallas University of Manchester (Uo. M) ICM Warsaw (ICM) Deutscher Wetterdienst (DWD) University of Southampton (Soton) Fujitsu European Laboratories Argonne National Laboratories (ANL) Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 19

GRIP Status • UNICORE Interoperability Software implemented (FZJ) • Interoperability UNICORE – Globus demonstrated • Development of OGSA interfaces in 2003 Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 20

UNICORE Architecture List of Sites User Certificate Preparation and Control of jobs UNICORE Client Job Preparation Agent (JPA) Job Monitor Controller (JMC) AJO Unsafe Internet (SSL/https) Abstract status request optional firewall User authentication Site-specific authorization User mapping, Resources info Network Job Superviser (NJS) Job incarnation IDB Incarnated job in der Helmholtz-Gemeinschaft optional firewall Sending jobs to other gateways, data transfer NJS IDB UUDB . . . UUDB IDB Status request files Batch Subsystem Forschungszentrum Jülich UNICORE Gateway Safe Intranet (TCP/http) Target System Interface (TSI) Commands FZJ UNICORE Gateway TSI . . . TSI SV 1 Globus TSI T 3 E Batch Subsystem HPN Workshop via AG Globus 21

UNICORE & OGSI • Work to be completed in 2003 – Integrate Grid Service Hosting into Gateway – UPL port. Type • Vsite Discovery • Consign Job • Retrieve Outcome – Add Grid Service interface to Client – Broker Interfaces and Ontology Resource Broker User certificate security domain Unicore Client Resource Database Prime. Power Unicore Gateway Network Job Supervisor Organic Server OGSA Service User Database Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG Fujitsu Laboratories of Europe © 2003 User proxy certificate security domain 22

Future Work Full OGSA compliant UNICORE Implementation Resource Broker User certificate security domain Resource Database Prime. Power Unicore Client Unicore Gateway Network Job Supervisor Organic Server OGSA Service User Database Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG Fujitsu Laboratories of Europe © 2003 User proxy certificate security domain 23

Further Information • • • http: //www. unicore. org http: //www. pallas. com http: //www. eurogrid. org http: //www. grid-interoperability. org http: //www. gridstart. org http: //www. fz-juelich. de Forschungszentrum Jülich in der Helmholtz-Gemeinschaft HPN Workshop via AG 24