EGI-In. SPIRE The Auger VRC Jiří Chudoba Institute of Physics and CESNET, Prague The VO auger manager EGI-In. SPIRE RI-261323 www. egi. eu
Overview • • PAO Motivations for the grid computing Production experience Expectations from the VRC Steps to the Mo. U signature 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 2 www. egi. eu
Pierre Auger Observatory Collaboration • About 500 members from more than 111 institutes from 18 countries Participating countries: Argentina, Australia, Bolivia, Brazil, Czech Republic, France, Germany, Italy, Mexico, Netherlands, Poland, Portugal, Slovenia, Spain, United Kingdom, USA and Vietnam 7. 11. 2011 Jiri. Chudoba@cern. ch EGI-In. SPIRE RI-261323 3 www. egi. eu
Motivations for the GRID • Monte Carlo simulations of showers • CORSIKA program with several models • Tens of CPU hours for highest energies • Different energy bins, angles, models, chemical composition • Simulated data distribution • Uniform access to resources • Increase available resources NB: No real data on the grid – local sites (CCIN 2 P 3 and Fermilab) are sufficient 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 4 www. egi. eu
Usage Production team from Granada university – 4 people 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 5 www. egi. eu
Usage VO auger was the biggest EGI grid user after the 4 LHC VOs 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 6 www. egi. eu
Experience • Many job slots available • Storage shortage • Overfilling of some sites • High level tools were missing • Now we have second generation of custom tools for semiautomatic production • Reliability increases • Requires continuous effort 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 7 www. egi. eu
Experience • Grid access too complicated for some users • Certificate; UI ready for all VO users; • Certificate change – file ownership problem • Copies of output files also available via SRB, accessible with the local account in CC IN 2 P 3 • Some users very active • Lots of jobs, lots of small files 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 8 www. egi. eu
Expectations from the VRC • Forum for placing requirements • Better documentation (from both sides) • Custom procedures • SLA with sites, different policies • Additional manpower for solving concrete problems • Easier negotiations with sites 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 9 www. egi. eu
Some requirements • Data management • Mark an SE (a disk server, a FS) temporary unavailable – with proper error propagation to clients • Requires changes in clients too • Include time interval? • Use cases: failed storage components, wait for recovery, site maintenance, network failure, SE overloaded 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 10 www. egi. eu
More requirements examples • Global overview of the VO members activities • Resource usage accounting • Regular automatic overviews sent to the manager • Dashboard or Monalisa • Middleware including pilot factories and DBs for production jobs 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 11 www. egi. eu
Procedure towards the VRC • PAO management in favour • Suggested steps • A committee for “grid activities” • Define responsibilities within the PAO • Mo. U to be accepted by the PAO collaboration board (next meeting in November) 7. 11. 2011 EGI-In. SPIRE RI-261323 Jiri. Chudoba@cern. ch 12 www. egi. eu
Скачать презентацию EGI-In SPIRE The Auger VRC Jiří Chudoba Institute
afff875a7b1a6641ee5fd42131f36c31.ppt