895067cfe3933f3075a58f7d18aadd92.ppt
- Количество слайдов: 30
Enabling Grids for E-scienc. E Introduction to Grid Technologies in EGEE Emanouil Atanassov, Aneta Karaivanova and Todor Gurov Institute for Parallel Processing - BAS www. eu-egee. org INFSO-RI-508833 • i. ASTRO MC MEETING&WORKSHOP, 27 -30, APRIL, 2005, SOFIA, BULGARIA
Overview Enabling Grids for E-scienc. E • • Evolvement of Grids What is Grid? Grid Services Goals of the EGEE project Building a production Grid for e-Science Grid applications in EGEE and SEE-GRID The Grid Challenges INFSO-RI-508833 Grid Day Nis 31 Jan 2006 2
Evolvement of Grids Enabling Grids for E-scienc. E Historical perspective • Local Computing – All computing resources at single site. – People move to resources to work. • Remote Computing – Resources accessible from distance. – All significant resources still centralized. • Distributed Computing – Resources geographically distributed. – Specialized access; largely data transfers. • Grid Computing – Resources and services geographically distributed. – Standard interfaces; transfers of computations and data. • Web Services and Grid Computing – Grid Services – Industry adopts Grid technology INFSO-RI-508833 Grid Day Nis 31 Jan 2006 3
What is GRID? Enabling Grids for E-scienc. E “Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I. Foster) – Resources are controlled by their owners – The Grid infrastructure provides access to collaborators A Virtual Organization is: – People from different institutions working to solve a common goal – Sharing distributed processing and data resources Enabling People to Work Together on Challenging Projects – Science, Engineering, Medicine… - e-Science, e-Health – Public service, commerce… - e-Government, e-Business The Grid could be the “new age” Internet – ‘[The Grid] intends to make access to computing power, scientific data repositories and experimental facilities as easy as the Web makes access to information. ’, UK PM, 2002 INFSO-RI-508833 Grid Day Nis 31 Jan 2006 4
The GRID vision Enabling Grids for E-scienc. E • On one hand: – Researchers/employees perform their activities regardless of geographical location, interact with colleagues, share and access data • On the other hand: – Scientific instruments and experiments provide huge amount of data, incl. national databases • And in the middle: – The Grid: networked data, processing centres and ”grid middleware” as the “glue” of resources. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 5
Grid Services Enabling Grids for E-scienc. E • Basic unit of computation – job • Basic unit of storage – file • Information systems – BDII, Globus-mds, R-GMA, file catalogues, metadata catalogues • Authorization, authentication, accounting (AAA)– based on PKI (Public key infrastructure) • Every Grid site provides basic Grid services • Advanced Grid Services: MPI jobs, Mass Storage Facilities accessed via SRM, Fine grained AAA (VOMS, DGAS). INFSO-RI-508833 Grid Day Nis 31 Jan 2006 6
Grid Services - schema Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 7
Grid Services in g. Lite Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 8
EGEE Partner Federations Enabling Grids for E-scienc. E All work in EGEE will be carried out by the 70 partners grouped in 12 federations. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 9
Goals of the EGEE project Enabling Grids for E-scienc. E • Goal in one sentence: – Allow scientists from multiple domains to use, share, and manage geographically distributed resources transparently. • The EGEE project brings together experts from over 27 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure, available to scientists 24 hours-aday. • The project aims to provide researchers in academia and industry with access to major computing resources, independent of their geographic location. The EGEE project will also focus on attracting a wide range of new users to the Grid. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 10
Scientific disciplines to run Grid applications Enabling Grids for E-scienc. E • EGEE aims to establish production quality sustained Grid services – 3000 users from at least 5 disciplines – integrate 50 sites into a common infrastructure – offer 5 Petabytes (1015) storage • Demonstrate a viable general process to bring other scientific communities on board INFSO-RI-508833 Pilot New Grid Day Nis 31 Jan 2006 11
EGEE – building a production Grid for e-Science Enabling Grids for E-scienc. E • • Operations Management Centre (OMC): – At CERN – coordination etc Core Infrastructure Centres (CIC) – Manage daily grid operations – oversight, troubleshooting Run essential infrastructure services Provide 2 nd level support to ROCs UK/I, Fr, It, CERN, + Russia (M 12) Taipei also run a CIC • • INFSO-RI-508833 – – Regional Operations Centres (ROC) – Act as front-line support for user and operations issues – Provide local knowledge and adaptations – One in each region – many distributed User Support Centre (GGUS) – In FZK – manage PTS – provide single point of contact (service desk) – Not foreseen as such in TA, but need is clear Grid Day Nis 31 Jan 2006 12
Components of a production Grid Enabling Grids for E-scienc. E • A production Grid consists of stable interoperating Grid sites (Resource centres), which enable access to Grid users from various Virtual Organizations • Every Grid site provides basic Grid services and follows strict operational procedures. • Monitoring allows fast detection of problems and their resolution or isolation. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 13
BG 01 -IPP setup Enabling Grids for E-scienc. E enter Grid Terminals enter Grid CE SE BDII UI UI - PKI X. 509 certificate keys - JDL files WN WN WN RB/II INFSO-RI-508833 Grid Day Nis 31 Jan 2006 14
Structure of EGEE operations Enabling Grids for E-scienc. E • • RC RC ROC RC CIC RC RC – – – OMC CIC RC RC RC ROC RC RC = Resource Centre INFSO-RI-508833 • ROC CIC RC The grid is flat, but Hierarchy of responsibility – Essential to scale the operation CICs act as a single Operations Centre – Operational oversight (grid RC ROC RC RC • operator) responsibility rotates weekly between CICs Report problems to ROC/RC ROC is responsible for ensuring problem is resolved ROC oversees regional RCs – ROCs responsible for organising the operations in a region – Coordinate deployment of middleware, etc CERN coordinates sites not associated with a ROC Grid Day Nis 31 Jan 2006 15
Operations monitoring maps Enabling Grids for E-scienc. E • In LCG-2: • 137 sites, 34 countries • >12, 000 cpu • ~5 PB storage • Includes non-EGEE sites: • 9 countries, 18 sites INFSO-RI-508833 Grid Day Nis 31 Jan 2006 16
Selection of Monitoring tools Enabling Grids for E-scienc. E GIIS Monitor graphs Sites Functional Tests GOC Data Base Scheduled Downtimes Grid. Ice – VO view Grid. Ice – fabric view Live Job Monitor Certificate Lifetime Monitor Note: Those thumbnails are links and are clickable. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 17
Example: LHC at CERN Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 18
CMS LHC Experiment Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 19
Example biomedical app: g. PTM 3 D Enabling Grids for E-scienc. E • One data set is – DICOM files: 100 MB – 1 GB – One radiological image: 20 MB – 500 MB • Complex interface: optimized graphics and medicallyoriented interactions • Physician interaction is required at and inside all steps Poorly discriminant data, pathologies, medical windowing Interaction Acquire INFSO-RI-508833 Explore Analyse Interpret Render Grid Day Nis 31 Jan 2006 20
Figures Enabling Grids for E-scienc. E Dataset Input data Output data Tasks 87 MB 3 MB 6 MB 18 KB/slice 106 KB/slice 169 5 min 15 s 1 min 54 s 37 s 18 s Medium 210 MB body 9. 6 MB 57 MB 25 KB/slice 151 KB/slice 378 33 min 11 min 5 s 2 min 30 s 1 min 15 s Large body 346 MB 15 MB 86 MB 22 KB/sclice 131 KB/slice 676 18 min 2 min 03 Lungs 87 MB 410 KB 4 KB/slice 36 s 24 s Small body INFSO-RI-508833 2. 3 MB 24 KB/slice 95 Standalone EGEE Execution 14 procs. Grid Day Nis 31 Jan 2006 21
Example: The MAGIC Telescope Enabling Grids for E-scienc. E • Ground based Air Cerenkov Telescope • Gamma ray: 30 Ge. V - Te. V • La. Palma, Canary Islands (28° North, 18° West) • 17 m diameter • operation since autumn 2003 (still in commissioning) • Collaborators: IFAE Barcelona, UAB Barcelona, Humboldt U. Berlin, UC Davis, U. Lodz, UC Madrid, MPI München, INFN / U. Padova, U. Potchefstrom, INFN / U. Siena, Tuorla Observatory, INFN / U. Udine, U. Würzburg, Yerevan Physics Inst. , ETH Zürich Physics Goals: Origin of VHE Gamma rays Active Galactic Nuclei Supernova Remnants Unidentified EGRET sources Gamma Ray Burst INFSO-RI-508833 Grid Day Nis 31 Jan 2006 22
Ground based γ-ray astronomy Enabling Grids for E-scienc. E GLAST (~ 1 m 2) Gamma ray Particle shower Cherenkov light Image of particle shower in telescope camera ~ 1 o Che ren k ov l igh t ~ 10 km ~ 120 m INFSO-RI-508833 reconstruct: arrival direction, energy reject hadron background Grid Day Nis 31 Jan 2006 23
MAGIC – Hadron rejection Enabling Grids for E-scienc. E • Based on extensive Monte Carlo Simulation – air shower simulation program CORSIKA – Simulation of hadronic background is very CPU consuming § to simulate the background of one night, 70 CPUs (P 4 2 GHz) needs to run 19200 days § to simulate the gamma events of one night for a Crab like source takes 288 days. – At higher energies (> 70 Ge. V) observations are possible already by On-Off method (This reduces the On-time by a factor of two) – Lowering the threshold of the MAGIC telescope requires new methods based on Monte Carlo Simulations INFSO-RI-508833 Grid Day Nis 31 Jan 2006 24
BG application in SEE-GRID VO SALUTE Enabling Grids for E-scienc. E The Problem: ultra-fast semiconductor carrier transport femtosecond relaxation of hot electrons by phonon emission in presence of electric field. Barker-Ferry equation and Monte Carlo approach • Application in nanotechnologies: innovative results for Ga. As: collision broadening and memory effects of quantum kinetic model; Intra-collision field effect: quantum scattering retarding and accelerating field. • “NP-hard” problem concerning the evolution time • Parallel and Grid implementation INFSO-RI-508833 Grid Day Nis 31 Jan 2006 25
Wigner function Enabling Grids for E-scienc. E 800 x 260 points 150 fs INFSO-RI-508833 Grid Day Nis 31 Jan 2006 26
Energy relaxation process: Enabling Grids for E-scienc. E collisional broadening Accumulation From 10 fs up to 250 fs INFSO-RI-508833 Grid Day Nis 31 Jan 2006 27
BG application in ESR VO – air pollution prediction Enabling Grids for E-scienc. E • Under development by Tzvetan Ostromsky from IPP • Transition from HPC to Grid computing INFSO-RI-508833 Grid Day Nis 31 Jan 2006 28
Challenges before new sites Enabling Grids for E-scienc. E • Install middleware and follow security and middleware upgrades in a timely fashion • Present valuable resource to the Virtual Organizations that the site supports • Participate in the various challenges. So far we have seen the HEP and the Biomed VO challenges, and the security challenges • Participate in innovation efforts – development of middleware and/or grid applications • Attract new users • The Grid is about people INFSO-RI-508833 Grid Day Nis 31 Jan 2006 29
BG Grid support centre contact information Enabling Grids for E-scienc. E Contact persons: • Emanouil Atanassov, SA 1 Activity Leader, emanouil@parallel. bas. bg • Aneta Karaivanova, NA 2 Activity Leader, anet@parallel. bas. bg • Todor Gurov, Alternate EGEE SEE-ROC and SEE-GRID manager, gurov@parallel. bas. bg • Ivan Dimov, EGEE & SEE-GRID Project manager for BG ivdimov@bas. bg • http: //www. grid. bas. bg/ INFSO-RI-508833 Grid Day Nis 31 Jan 2006 30
895067cfe3933f3075a58f7d18aadd92.ppt