Скачать презентацию Enabling Grids for E-scienc E Introduction to Grid Скачать презентацию Enabling Grids for E-scienc E Introduction to Grid

895067cfe3933f3075a58f7d18aadd92.ppt

  • Количество слайдов: 30

Enabling Grids for E-scienc. E Introduction to Grid Technologies in EGEE Emanouil Atanassov, Aneta Enabling Grids for E-scienc. E Introduction to Grid Technologies in EGEE Emanouil Atanassov, Aneta Karaivanova and Todor Gurov Institute for Parallel Processing - BAS www. eu-egee. org INFSO-RI-508833 • i. ASTRO MC MEETING&WORKSHOP, 27 -30, APRIL, 2005, SOFIA, BULGARIA

Overview Enabling Grids for E-scienc. E • • Evolvement of Grids What is Grid? Overview Enabling Grids for E-scienc. E • • Evolvement of Grids What is Grid? Grid Services Goals of the EGEE project Building a production Grid for e-Science Grid applications in EGEE and SEE-GRID The Grid Challenges INFSO-RI-508833 Grid Day Nis 31 Jan 2006 2

Evolvement of Grids Enabling Grids for E-scienc. E Historical perspective • Local Computing – Evolvement of Grids Enabling Grids for E-scienc. E Historical perspective • Local Computing – All computing resources at single site. – People move to resources to work. • Remote Computing – Resources accessible from distance. – All significant resources still centralized. • Distributed Computing – Resources geographically distributed. – Specialized access; largely data transfers. • Grid Computing – Resources and services geographically distributed. – Standard interfaces; transfers of computations and data. • Web Services and Grid Computing – Grid Services – Industry adopts Grid technology INFSO-RI-508833 Grid Day Nis 31 Jan 2006 3

What is GRID? Enabling Grids for E-scienc. E “Coordinated resource sharing and problem solving What is GRID? Enabling Grids for E-scienc. E “Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations” (I. Foster) – Resources are controlled by their owners – The Grid infrastructure provides access to collaborators A Virtual Organization is: – People from different institutions working to solve a common goal – Sharing distributed processing and data resources Enabling People to Work Together on Challenging Projects – Science, Engineering, Medicine… - e-Science, e-Health – Public service, commerce… - e-Government, e-Business The Grid could be the “new age” Internet – ‘[The Grid] intends to make access to computing power, scientific data repositories and experimental facilities as easy as the Web makes access to information. ’, UK PM, 2002 INFSO-RI-508833 Grid Day Nis 31 Jan 2006 4

The GRID vision Enabling Grids for E-scienc. E • On one hand: – Researchers/employees The GRID vision Enabling Grids for E-scienc. E • On one hand: – Researchers/employees perform their activities regardless of geographical location, interact with colleagues, share and access data • On the other hand: – Scientific instruments and experiments provide huge amount of data, incl. national databases • And in the middle: – The Grid: networked data, processing centres and ”grid middleware” as the “glue” of resources. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 5

Grid Services Enabling Grids for E-scienc. E • Basic unit of computation – job Grid Services Enabling Grids for E-scienc. E • Basic unit of computation – job • Basic unit of storage – file • Information systems – BDII, Globus-mds, R-GMA, file catalogues, metadata catalogues • Authorization, authentication, accounting (AAA)– based on PKI (Public key infrastructure) • Every Grid site provides basic Grid services • Advanced Grid Services: MPI jobs, Mass Storage Facilities accessed via SRM, Fine grained AAA (VOMS, DGAS). INFSO-RI-508833 Grid Day Nis 31 Jan 2006 6

Grid Services - schema Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Grid Services - schema Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 7

Grid Services in g. Lite Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis Grid Services in g. Lite Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 8

EGEE Partner Federations Enabling Grids for E-scienc. E All work in EGEE will be EGEE Partner Federations Enabling Grids for E-scienc. E All work in EGEE will be carried out by the 70 partners grouped in 12 federations. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 9

Goals of the EGEE project Enabling Grids for E-scienc. E • Goal in one Goals of the EGEE project Enabling Grids for E-scienc. E • Goal in one sentence: – Allow scientists from multiple domains to use, share, and manage geographically distributed resources transparently. • The EGEE project brings together experts from over 27 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure, available to scientists 24 hours-aday. • The project aims to provide researchers in academia and industry with access to major computing resources, independent of their geographic location. The EGEE project will also focus on attracting a wide range of new users to the Grid. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 10

Scientific disciplines to run Grid applications Enabling Grids for E-scienc. E • EGEE aims Scientific disciplines to run Grid applications Enabling Grids for E-scienc. E • EGEE aims to establish production quality sustained Grid services – 3000 users from at least 5 disciplines – integrate 50 sites into a common infrastructure – offer 5 Petabytes (1015) storage • Demonstrate a viable general process to bring other scientific communities on board INFSO-RI-508833 Pilot New Grid Day Nis 31 Jan 2006 11

EGEE – building a production Grid for e-Science Enabling Grids for E-scienc. E • EGEE – building a production Grid for e-Science Enabling Grids for E-scienc. E • • Operations Management Centre (OMC): – At CERN – coordination etc Core Infrastructure Centres (CIC) – Manage daily grid operations – oversight, troubleshooting Run essential infrastructure services Provide 2 nd level support to ROCs UK/I, Fr, It, CERN, + Russia (M 12) Taipei also run a CIC • • INFSO-RI-508833 – – Regional Operations Centres (ROC) – Act as front-line support for user and operations issues – Provide local knowledge and adaptations – One in each region – many distributed User Support Centre (GGUS) – In FZK – manage PTS – provide single point of contact (service desk) – Not foreseen as such in TA, but need is clear Grid Day Nis 31 Jan 2006 12

Components of a production Grid Enabling Grids for E-scienc. E • A production Grid Components of a production Grid Enabling Grids for E-scienc. E • A production Grid consists of stable interoperating Grid sites (Resource centres), which enable access to Grid users from various Virtual Organizations • Every Grid site provides basic Grid services and follows strict operational procedures. • Monitoring allows fast detection of problems and their resolution or isolation. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 13

BG 01 -IPP setup Enabling Grids for E-scienc. E enter Grid Terminals enter Grid BG 01 -IPP setup Enabling Grids for E-scienc. E enter Grid Terminals enter Grid CE SE BDII UI UI - PKI X. 509 certificate keys - JDL files WN WN WN RB/II INFSO-RI-508833 Grid Day Nis 31 Jan 2006 14

Structure of EGEE operations Enabling Grids for E-scienc. E • • RC RC ROC Structure of EGEE operations Enabling Grids for E-scienc. E • • RC RC ROC RC CIC RC RC – – – OMC CIC RC RC RC ROC RC RC = Resource Centre INFSO-RI-508833 • ROC CIC RC The grid is flat, but Hierarchy of responsibility – Essential to scale the operation CICs act as a single Operations Centre – Operational oversight (grid RC ROC RC RC • operator) responsibility rotates weekly between CICs Report problems to ROC/RC ROC is responsible for ensuring problem is resolved ROC oversees regional RCs – ROCs responsible for organising the operations in a region – Coordinate deployment of middleware, etc CERN coordinates sites not associated with a ROC Grid Day Nis 31 Jan 2006 15

Operations monitoring maps Enabling Grids for E-scienc. E • In LCG-2: • 137 sites, Operations monitoring maps Enabling Grids for E-scienc. E • In LCG-2: • 137 sites, 34 countries • >12, 000 cpu • ~5 PB storage • Includes non-EGEE sites: • 9 countries, 18 sites INFSO-RI-508833 Grid Day Nis 31 Jan 2006 16

Selection of Monitoring tools Enabling Grids for E-scienc. E GIIS Monitor graphs Sites Functional Selection of Monitoring tools Enabling Grids for E-scienc. E GIIS Monitor graphs Sites Functional Tests GOC Data Base Scheduled Downtimes Grid. Ice – VO view Grid. Ice – fabric view Live Job Monitor Certificate Lifetime Monitor Note: Those thumbnails are links and are clickable. INFSO-RI-508833 Grid Day Nis 31 Jan 2006 17

Example: LHC at CERN Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Example: LHC at CERN Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 18

CMS LHC Experiment Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan CMS LHC Experiment Enabling Grids for E-scienc. E INFSO-RI-508833 Grid Day Nis 31 Jan 2006 19

Example biomedical app: g. PTM 3 D Enabling Grids for E-scienc. E • One Example biomedical app: g. PTM 3 D Enabling Grids for E-scienc. E • One data set is – DICOM files: 100 MB – 1 GB – One radiological image: 20 MB – 500 MB • Complex interface: optimized graphics and medicallyoriented interactions • Physician interaction is required at and inside all steps Poorly discriminant data, pathologies, medical windowing Interaction Acquire INFSO-RI-508833 Explore Analyse Interpret Render Grid Day Nis 31 Jan 2006 20

Figures Enabling Grids for E-scienc. E Dataset Input data Output data Tasks 87 MB Figures Enabling Grids for E-scienc. E Dataset Input data Output data Tasks 87 MB 3 MB 6 MB 18 KB/slice 106 KB/slice 169 5 min 15 s 1 min 54 s 37 s 18 s Medium 210 MB body 9. 6 MB 57 MB 25 KB/slice 151 KB/slice 378 33 min 11 min 5 s 2 min 30 s 1 min 15 s Large body 346 MB 15 MB 86 MB 22 KB/sclice 131 KB/slice 676 18 min 2 min 03 Lungs 87 MB 410 KB 4 KB/slice 36 s 24 s Small body INFSO-RI-508833 2. 3 MB 24 KB/slice 95 Standalone EGEE Execution 14 procs. Grid Day Nis 31 Jan 2006 21

Example: The MAGIC Telescope Enabling Grids for E-scienc. E • Ground based Air Cerenkov Example: The MAGIC Telescope Enabling Grids for E-scienc. E • Ground based Air Cerenkov Telescope • Gamma ray: 30 Ge. V - Te. V • La. Palma, Canary Islands (28° North, 18° West) • 17 m diameter • operation since autumn 2003 (still in commissioning) • Collaborators: IFAE Barcelona, UAB Barcelona, Humboldt U. Berlin, UC Davis, U. Lodz, UC Madrid, MPI München, INFN / U. Padova, U. Potchefstrom, INFN / U. Siena, Tuorla Observatory, INFN / U. Udine, U. Würzburg, Yerevan Physics Inst. , ETH Zürich Physics Goals: Origin of VHE Gamma rays Active Galactic Nuclei Supernova Remnants Unidentified EGRET sources Gamma Ray Burst INFSO-RI-508833 Grid Day Nis 31 Jan 2006 22

Ground based γ-ray astronomy Enabling Grids for E-scienc. E GLAST (~ 1 m 2) Ground based γ-ray astronomy Enabling Grids for E-scienc. E GLAST (~ 1 m 2) Gamma ray Particle shower Cherenkov light Image of particle shower in telescope camera ~ 1 o Che ren k ov l igh t ~ 10 km ~ 120 m INFSO-RI-508833 reconstruct: arrival direction, energy reject hadron background Grid Day Nis 31 Jan 2006 23

MAGIC – Hadron rejection Enabling Grids for E-scienc. E • Based on extensive Monte MAGIC – Hadron rejection Enabling Grids for E-scienc. E • Based on extensive Monte Carlo Simulation – air shower simulation program CORSIKA – Simulation of hadronic background is very CPU consuming § to simulate the background of one night, 70 CPUs (P 4 2 GHz) needs to run 19200 days § to simulate the gamma events of one night for a Crab like source takes 288 days. – At higher energies (> 70 Ge. V) observations are possible already by On-Off method (This reduces the On-time by a factor of two) – Lowering the threshold of the MAGIC telescope requires new methods based on Monte Carlo Simulations INFSO-RI-508833 Grid Day Nis 31 Jan 2006 24

BG application in SEE-GRID VO SALUTE Enabling Grids for E-scienc. E The Problem: ultra-fast BG application in SEE-GRID VO SALUTE Enabling Grids for E-scienc. E The Problem: ultra-fast semiconductor carrier transport femtosecond relaxation of hot electrons by phonon emission in presence of electric field. Barker-Ferry equation and Monte Carlo approach • Application in nanotechnologies: innovative results for Ga. As: collision broadening and memory effects of quantum kinetic model; Intra-collision field effect: quantum scattering retarding and accelerating field. • “NP-hard” problem concerning the evolution time • Parallel and Grid implementation INFSO-RI-508833 Grid Day Nis 31 Jan 2006 25

Wigner function Enabling Grids for E-scienc. E 800 x 260 points 150 fs INFSO-RI-508833 Wigner function Enabling Grids for E-scienc. E 800 x 260 points 150 fs INFSO-RI-508833 Grid Day Nis 31 Jan 2006 26

Energy relaxation process: Enabling Grids for E-scienc. E collisional broadening Accumulation From 10 fs Energy relaxation process: Enabling Grids for E-scienc. E collisional broadening Accumulation From 10 fs up to 250 fs INFSO-RI-508833 Grid Day Nis 31 Jan 2006 27

BG application in ESR VO – air pollution prediction Enabling Grids for E-scienc. E BG application in ESR VO – air pollution prediction Enabling Grids for E-scienc. E • Under development by Tzvetan Ostromsky from IPP • Transition from HPC to Grid computing INFSO-RI-508833 Grid Day Nis 31 Jan 2006 28

Challenges before new sites Enabling Grids for E-scienc. E • Install middleware and follow Challenges before new sites Enabling Grids for E-scienc. E • Install middleware and follow security and middleware upgrades in a timely fashion • Present valuable resource to the Virtual Organizations that the site supports • Participate in the various challenges. So far we have seen the HEP and the Biomed VO challenges, and the security challenges • Participate in innovation efforts – development of middleware and/or grid applications • Attract new users • The Grid is about people INFSO-RI-508833 Grid Day Nis 31 Jan 2006 29

BG Grid support centre contact information Enabling Grids for E-scienc. E Contact persons: • BG Grid support centre contact information Enabling Grids for E-scienc. E Contact persons: • Emanouil Atanassov, SA 1 Activity Leader, [email protected] bas. bg • Aneta Karaivanova, NA 2 Activity Leader, [email protected] bas. bg • Todor Gurov, Alternate EGEE SEE-ROC and SEE-GRID manager, [email protected] bas. bg • Ivan Dimov, EGEE & SEE-GRID Project manager for BG [email protected] bg • http: //www. grid. bas. bg/ INFSO-RI-508833 Grid Day Nis 31 Jan 2006 30