The European Data Grid Project Fabrizio Gagliardi EU

The European Data. Grid Project Fabrizio Gagliardi EU Data. Grid Project Leader CERN fabrizio. gagliardi@cern. ch

The Grid metaphor Mobile Access Workstation G R I D M I D D L E W A R E Supercomputer, PC-Cluster Data-storage, Sensors, Experiments Visualising Internet, networks Fabrizio Gagliardi EDG 5 Piliscsaba

Technology Trends • Very fast networks can be built and are becoming much cheaper • Archival and disk storage becoming much cheaper • Sensors and instruments becoming ubiquitous and have much higher resolution => oceans of data • Computing power continues to increase – so many groups can analyze lots of data • Middleware, while still developing, is not vaporware • Everyone uses the web, so concept is not foreign • Some demonstration projects have shown the benefits (EU Data. Grid test bed among others) Fabrizio Gagliardi EDG 5 Piliscsaba

Grids Leverage Major Investments in Experimental Research Facilities • Research instruments, satellites, particle accelerators, etc. , cost a great deal • Data from those devices can be accessed analyzed by many more scientists from anywhere – not just the team that gathered the data or institutes from privileged countries • Easier international collaboration Fabrizio Gagliardi EDG 5 Piliscsaba

The EU Data. Grid Project • • • 21 Partners 9. 8 M Euros EU funding over 3 years 90% for middleware and applications (HEP, EO and Biomedical) Three year phased developments & demos (20012003) Spin-off: – Data. TAG (2002 -2003) – Cross. Grid (2002 -2004) – Grid. Start (2002 -2004) Fabrizio Gagliardi EDG 5 Piliscsaba

EU Data. Grid Project Objectives • Use Grid technology to develop a sustainable computing model for effective share of computing resources and data for large scientific communities • Specific project objectives: – Middleware for fabric & Grid management (mostly funded by the EU) – Large scale testbeds (mostly funded by the partners) – Production quality demonstrations (partially funded by the EU) Fabrizio Gagliardi EDG 5 Piliscsaba

EU Data. Grid Project Objectives • • • Collaborate with and complement other European and US projects Test and demonstrator of EU RN/Geant Contribute to Open Standards and international bodies: – Co-founder of Global GRID Forum and host of GGF 1 and GGF 3, co-founder of GRIDSTART • Dissemination – Industry and Research Forum for dissemination of project results • Education: Tutorials, CSC 02, Road Show Fabrizio Gagliardi EDG 5 Piliscsaba

Project Schedule • Project started on 1/1/2001 • Test. Bed 0 (early 2001) – International test bed 0 infrastructure deployed • Test. Bed 1. 0 ( March 2002 ) • Project successfully reviewed by EU on March 1 st 2002 • Test. Bed 1. 2 ( now ) – First release of EU Data. Grid software to defined users within the project: • HEP experiments, Earth Observation, Biomedical applications • Test. Bed 2 (end of 2002) – Builds on Test. Bed 1 to extend facilities of Data. Grid • Test. Bed 3 (Fall 2003) • Project completion expected by end 2003 Fabrizio Gagliardi EDG 5 Piliscsaba

Data. Grid Main Partners • CERN – International (Switzerland/France) • CNRS - France • ESA/ESRIN – International (Italy) • INFN - Italy • NIKHEF – The Netherlands • PPARC - UK Fabrizio Gagliardi EDG 5 Piliscsaba

Assistant Partners Industrial Partners • Datamat (Italy) • IBM-UK (UK) • CS-SI (France) Research and Academic Institutes • CESNET (Czech Republic) • Commissariat à l'énergie atomique (CEA) – France • Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) • Consiglio Nazionale delle Ricerche (Italy) • Helsinki Institute of Physics – Finland • Institut de Fisica d'Altes Energies (IFAE) - Spain • Istituto Trentino di Cultura (IRST) – Italy • Konrad-Zuse-Zentrum für Informationstechnik Berlin - Germany • Royal Netherlands Meteorological Institute (KNMI) • Ruprecht-Karls-Universität Heidelberg - Germany • Stichting Academisch Rekencentrum Amsterdam (SARA) – Netherlands • Swedish Research Council - Sweden Fabrizio Gagliardi EDG 5 Piliscsaba

EDG Highlights • The project has passed the first EU review (last March) – All 21 partners have contributed at least at contractual level – All EU deliverables (40, >2000 pages) submitted in time (available, code & documents via www. edg. org) – 60 EU funded FTEs and 106 un-funded FTEs – One of the largest Grid technology development and demonstrator • Real applications being deployed now Fabrizio Gagliardi EDG 5 Piliscsaba

Particle Physics Applications CMS Storage – ATLAS Raw recording rate 0. 1 – 1 GByte/sec Accumulating data at 8 -10 Peta. Bytes/year 10 Peta. Bytes of disk LHCb Processing – 100, 000 of today’s fastest PCs Fabrizio Gagliardi EDG 5 Piliscsaba

Biomedical applications • Data mining on genomic databases (exponential growth) • Indexing of medical databases (Tb/hospital/year) • Collaborative framework for large scale experiments (e. g. epidemiological studies) • Parallel processing for –Databases analysis –Complex 3 D modelling Fabrizio Gagliardi EDG 5 Piliscsaba

Earth Observations ESA missions: • about 100 Gbytes of data per day (ERS 1/2) • 500 Gbytes, for the next ENVISAT mission (launched March 1 st) EO requirements for the Grid: • enhance the ability to access high level products • allow reprocessing of large historical archives • improve Earth science complex applications (data fusion, data mining, modelling …) Fabrizio Gagliardi EDG 5 Piliscsaba

Data. Grid Testbed Sites(>40) HEP sites ESA sites Dubna Lund RAL Estec KNMI IPSL Paris Santander Lisboa CERN Moscow Berlin Prague Brno Lyon Grenoble Milano PD-LNL Torino BO-CNAF Madrid Marseille Pisa Barcelona ESRIN Roma Valencia Catania Francois. Etienne@in 2 p 3. fr EDG 5 Piliscsaba Fabrizio Gagliardi - Antonia. Ghiselli@cnaf. infn. it

Project Dissemination www. eu-datagrid. org CERN School of Computing 2002 Programme includes Grid Lectures by Ian Foster Carl Kesselman Hands-on tutorial Data. Grid Vico Equense, Italy, 15 -28 September 2002 The 2002 CERN School of Computing is organised by CERN, with the Institute of Composite and Biomedical Materials, National Research Council, Naples, Italy Fabrizio Gagliardi EDG 5 Piliscsaba

Future Plans • Concentrate on production quality and real applications • Complete the programme of work till end of 2003 • Educate new users and disseminate results • Make plans to conserve momentum and assets in the EU FP 6 Fabrizio Gagliardi EDG 5 Piliscsaba

A proposal to the EU • EDG has already demonstrated the viability of this technology • EU had a fast start in supporting 18 Grid projects in the last two years! • RN Geant offers an excellent basis for a large European Grid infrastructure • The EU FP 6 programme should encourage and support the deployment and production quality operation of a large international Grid infrastructure open to research and industry in Europe Fabrizio Gagliardi EDG 5 Piliscsaba

The EGEE Eo. I • EGEE: Integrated Infrastructure Initiative (III) to support ERA (European Research Area) 300 M € planned (100 M € requested) • Vision: to create and deploy Grid technologies to enable the widespread uptake of e-Science applications throughout the European Research Area • Four key objectives: – integrating Grid technological developments from across Europe; – establishing a Europe-wide Grid infrastructure for science and industry with a focus on heterogeneity and interoperability; – enabling the creation of e-Science applications from across the scientific and industrial spectrum; – ensuring the timely delivery of the project’s programme of work, guided by the needs of academic and industrial partners. Fabrizio Gagliardi EDG 5 Piliscsaba

The EGEE Eo. I continued • Approach: coordinated submission with RN Geant follow-up (200 M € requested) • Focus on the Grid production infrastructure to support vertical and horizontal applications (vertical: Bio, EO, PP, Engineering, Commerce? Industry? horizontal: CS and added value services) • Special emphasis of the EU programme on SMEs • Distributed managerial structure by coordinating national and regional Grid initiatives Fabrizio Gagliardi EDG 5 Piliscsaba

Final word… • EU Data. Grid is a good first step together with the other related Grid initiatives • Need not to lose momentum but aggregate interest and resources in science, industry and funding agencies for proposals in the EU FP 6 • Significant contribution from the Hungarian partners so far • Important step this week here to review the progress and make plans for the future Fabrizio Gagliardi EDG 5 Piliscsaba

Many thanks for the kind hospitality and excellent support! Fabrizio Gagliardi EDG 5 Piliscsaba