Скачать презентацию US-CMS User Facilities Project Vivian O Dell Fermilab USCMS Скачать презентацию US-CMS User Facilities Project Vivian O Dell Fermilab USCMS

2a7c107b07789ddd72f51bf5f8db21ce.ppt

  • Количество слайдов: 33

US-CMS User Facilities Project Vivian O’Dell Fermilab USCMS Software and Computing Oversight Panel Review US-CMS User Facilities Project Vivian O’Dell Fermilab USCMS Software and Computing Oversight Panel Review October 23, 2000

Goals of the US-CMS S&C Project u. To provide the software and computing resources Goals of the US-CMS S&C Project u. To provide the software and computing resources needed to enable US physicists to fully participate in the physics program of CMS u. Allow US physicists to play key roles and exert an appropriate level of leadership in all stages of computing related activities … èFrom software infrastructure to reconstruction to extraction of physics results èFrom their home institutions, as well as at CERN This capability should extend to physicists working at their home institutions October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 2

Data Grid Hierarchy (CMS) 1 TIPS = 25, 000 Spec. Int 95 PC (2000) Data Grid Hierarchy (CMS) 1 TIPS = 25, 000 Spec. Int 95 PC (2000) = 20 Spec. Int 95 ~PBytes/sec Online System Bunch crossing per 25 nsecs. 100 triggers per second Event is ~1 MByte in size Tier 1 France Regional Center Tier 0+1 CERN Computer Center 0. 6 -2. 5 Gbits/sec or Air Freight UK Regional Center Italy Regional Center Tier 2 FNAL Regional Center ~2. 4 Gbits/sec Tier 2 Center Tier 2 Center ~622 Mbits/sec Tier 3 Institute Physics data cache Institute 100 - 1000 Mbits/sec Workstations October 23, 2000 ~100 MBytes/sec Tier 4 Physicists work on analysis “channels”. Each institute has ~10 physicists working on one or more channels Data for these channels should be cached by the institute server Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 3

Issues èScale of LHC computing is greater than anything we’ve tried in HEP p. Issues èScale of LHC computing is greater than anything we’ve tried in HEP p. Number of collaborating physicists p. Number of collaborating institutions p. Number of collaborating nations p. Geographic distribution p. Size and complexity of the data set p. Projected length of the experiment èWe have never had a single site which has been capable of providing resources on the scale of the whole LHC effort -- CERN does not even plan to try èThere are excellent reasons to believe that the experience at our largest current sites will not scale or provide appropriate resources for the LHC October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 4

User Facility Project Provide the enabling infrastructure of software and computing for US physicists User Facility Project Provide the enabling infrastructure of software and computing for US physicists to fully participate in the CMS physics program è Acquire, develop, install, integrate, commission and operate the infrastructure è Includes a Tier 1 Regional Center at Fermilab èPlans are being developed for lower level centers and support for collaborators at their home institutions p Tier 2 = 10 -20% of Tier 1 èIncludes S/W training, workshops and support for USCMS physicists è High speed US and international networks are a major requirement p We assume that DOE/NSF want to address the issue of the CERNUS link in a combined way for CMS and ATLAS to get the most cost effective solution p We assume that the issue of connectivity of individual institutions to FNAL has to be worked out in an individual basis because there are policy issues at each institution October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 5

Schedule for US-CMS S&C Project In order to match the proposed funding profile from Schedule for US-CMS S&C Project In order to match the proposed funding profile from DOE/NSF more closely as well as the LHC schedule, we have stretched out the CMS Software and Computing Project Schedule. Now – end FY 2003: R&D phase FY 2004 -FY 2006: Implementation Phase FY 2007 and onwards: Operations Phase This required a new hardware cost estimate and resource loaded WBS. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 6

Hoffmann Review u To answer to computing and software needs, CERN has set up Hoffmann Review u To answer to computing and software needs, CERN has set up a review of LHC software and computing è Chaired by Hans Hoffmann, Director for Scientific Computing u Three Panels: è Software: quantify software needs for each of the LHC experiments è Computing Model: review the computing model for each experiment and make recommendations è Resource panel: review computing resources required by all LHC experiments u Resource group focuses on the amount of computing needed at CERN and at Regional Centers p Eventual result: MOUs between T 1 RC’s and T 0 (CERN)? p This should represent the minimum computing requirements for a Regional Center October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 7

Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! Tape Storage at each Tier 1 for CMS Simulated event data 0. 2 PB General ESD’s 0. 2 PB Simulated ESD+AOD for analysis Raw Event Data 0. 04 PB 0. 05 PB (5% of yearly raw data) Reprocessed and Revised 0. 5 PB ESD’s AOD, TAG, Calibration, 0. 03 PB etc. Total 1. 02 PB Our T 1 h/w projections include robotics for 2 PB but media for 1 PB October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 8

Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! Disk Storage at each Tier 1 for CMS Simulated ESD + AOD 30 TB Calibration Data 10 TB Two versions of ESD for local analysis AOD, TAG, etc. 80 TB Disk Cache for Active Tape Total 130 (20%*active tape) 20 TB 270 TB Disk is assumed to be 75% efficient – approx RAID overhead or duplicate disk needs for I/O. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 9

Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! Hoffmann Review Results from Hoffmann Review Precise numbers still in flux, but converging rapidly! CPU at each Tier 1 for CMS Simulation CPU Reprocessing CPU Analysis CPU * AMS CPU (I. e. CPU for data servers) Total 26 k Si 95’s Efficiency Factor = 80% 80 Si 95’s Efficiency Factor = 80% 60 k Si 95’s Efficiency Factor = 60% 12 k Si 95’s = 20% Analysis CPU Efficiency Factor = 100% 178 k Si 95’s * Not included in Hoffman Review October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 10

Hardware Costing Projections: CPU Commodity (Cheap) CPU • Use “Moore’s Law” to extrapolate from Hardware Costing Projections: CPU Commodity (Cheap) CPU • Use “Moore’s Law” to extrapolate from FNAL experience • Performance of CPU doubles every 1. 2 years while the cost stays the same. SMP (expensive) CPU Performance of CPU/Cost doubles every 1. 1 years October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 11

Hardware Costing Projections: Disk Commodity (Cheap) Disks • Performance/cost of disk doubles every 1. Hardware Costing Projections: Disk Commodity (Cheap) Disks • Performance/cost of disk doubles every 1. 4 years Datacenter (expensive) Disks Performance/cost of disk doubles every 1. 4 years October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 12

Hardware Costing Projections: Tape Media • Performance/cost of tape doubles every 2. 1 years Hardware Costing Projections: Tape Media • Performance/cost of tape doubles every 2. 1 years (? – not impressively linear…) Tape Drives Performance/cost of tape drives doubles every 2. 1 years I believe there is considerable risk in depending on tape storage over The next 15 years… We should be flexible here. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 13

Hardware Costing For UF w/o T 2 Hardware costing for User Facilities, excluding Tier Hardware Costing For UF w/o T 2 Hardware costing for User Facilities, excluding Tier 2 October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 14

Hardware Costing For UF Hardware costing for User Facilities, INCLUDING Tier 2 from Gri. Hardware Costing For UF Hardware costing for User Facilities, INCLUDING Tier 2 from Gri. Phy. N October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 15

Hardware Costing For UF Total hardware costs including escalation, overhead and 10% Management Reserve Hardware Costing For UF Total hardware costs including escalation, overhead and 10% Management Reserve October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 16

Personnel for the User Facility Project Estimating the number of people needed for the Personnel for the User Facility Project Estimating the number of people needed for the User Facility Project We did this in two ways: è“Top Down” using our experience with Run 2 è“Bottoms folks October 23, 2000 Up” developing a full WBS with experienced FNAL Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 17

“Top Down” Approach What is the appropriate scale for US-CMS User Facility? We can “Top Down” Approach What is the appropriate scale for US-CMS User Facility? We can ask Fermilab because: u. Scale of US Physicist Participation in CMS is comparable to CDF or D 0 during Run 2, e. g. ~500 physicists. This will most likely grow as LHC comes closer. u. Software and Computing for Run 2 was reviewed separately from the detector Run 2 project. There are good reasons for this. As a result we have good records as to the hardware and personnel spending for Run 2. u. We can start with a “top down” approach as to resources it took for CDF/D 0 and assume supporting roughly the same population with similar needs will take roughly the same amount of computing. u. This top down approach breaks down with respect to computing hardware, since LHC events are much more complicated (I. e. ~20 interactions/crossing for LHC, ~3 for Run 2). October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 18

WBS From Top Down 2001 2002 2003 1. 1 Tier 1 Regional Center 20 WBS From Top Down 2001 2002 2003 1. 1 Tier 1 Regional Center 20 17 1. 1. 1 Hardware Systems 9 7 1. 1. 2 Software Systems 11 10 1. 2 System and User Support 6 8 1. 3 Maintenance and Operation 5 6 1. 4 Support for Tier 2 RCs 2 2 1. 5 Networking 0. 5 1. 6 R&D and CMS Construction Phase Activities 6 7 1. 4. 1 Hardware 3 0 1. 4. 2 Software 3 4 October 23, 2000 2004 2005 2006 2007 8 18 4 9 1. 5 2 3. 5 5 1 1. 5 3 5 . 5 1 1. 5 2 1 1 2 2 2 7 3 6 3 3 3 0 2 0 0 4 3 1 0 0 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 19

Bottoms Up Approach We can also get the experts from Fermilab to help us Bottoms Up Approach We can also get the experts from Fermilab to help us estimate personnel and hardware costs for User Facility Computing. We first met to debate the WBS items and discuss the scope and schedule of the project. Then we assigned WBS items to experts in the area and had them start from a bottoms up approach without worrying about total FTEs from any one WBS item. People working on the WBS were from CDF/D 0/Networking/Systems Support and other CD areas of the lab. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 20

From “Bottoms Up” Approach 2001 2002 2003 2004 1. 1 Tier 1 Regional Center From “Bottoms Up” Approach 2001 2002 2003 2004 1. 1 Tier 1 Regional Center 20 18 1. 2 System and User Support 1. 3 4. 5 1. 3 Operations and Infrastructure 1 4. 5 1. 4 Tier 2 RCs 2. 5 3. 5 4 1. 5 Networking 0. 5 1. 6 Computing and Software R&D 3 1. 7 Construction Phase Computing 1. 8 Support of FNAL 2 based Computing 2005 2006 2007 13 17 2. 25 3 3 5 1. 25 3 5 5 6. 5 7. 5 1 2 2. 5 3 3 3 4 2 4. 5 2 1. 4 2 0. 6 0. 25 0. 6 0. 4 1 2. 6 2. 25 2 User Facilities (total) 10 15 18 30 37 41 October 23, 2000 Vivian O’Dell, US 7. 5 (Without T 2 Personnel) CMS User Facilities Project, USCMS S&C Oversight Panel Review 11. 5 14 25 30 34 43 36 21

Scale of User Facility Project Amazingly enough, the bottoms up approach gave us very Scale of User Facility Project Amazingly enough, the bottoms up approach gave us very similar results! I didn’t tell the experts in advance how many FTE’s I thought things would take because I wanted to see if there was any value to either top down or bottoms up. Considering they landed within 5% of each other, except for some scheduling differences, it gives me confidence that the scale is correct. Of course, although through the bottoms up approach we have assigned each FTE to tasks in detail, the project must maintain flexibility in order to succeed. For example, I assume the problem of large linux farms for interactive and “chaotic” analyses will be solved by 2004. If not we may have to buy different (expensive!) hardware or hire smarter people. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 22

User Facility Personnel Needs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, User Facility Personnel Needs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 23

Personnel Costs for User Facility Project Lots of different kinds of people with different Personnel Costs for User Facility Project Lots of different kinds of people with different skills are included in the WBS. We used average Computing Division salaries for Computing Professionals to calculate personnel costs. * Personnel Costs were calculated by: è Average CD/CP Salary = $78 k è Benefits = 28. 5% è FNAL Overhead = 30% è Add Travel+Desktop and general infrastructure support è Total encumbered salary = $154, 000 è (This gets escalated by 3%/year by usual DOE rules) * This average salary in CD includes high level people. Hopefully we will be hiring more of them in the future! October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 24

User Facility Personnel Costs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, User Facility Personnel Costs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 25

Total User Facility Costs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, Total User Facility Costs October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 26

Current Status and Recent Activities User Facility Current Status October 23, 2000 Vivian O’Dell, Current Status and Recent Activities User Facility Current Status October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 27

User Facility Personnel Have Hired 4 People this Year We continue to leverage additional User Facility Personnel Have Hired 4 People this Year We continue to leverage additional support from Fermilab when possible User Facility People èJoe Kaiser (WBS 1. 2/1. 7) system installation and commisioning; system administration p èYujun Wu (WBS 1. 7) p Computing simulation tools and verification for distributed computing R&D; documentation; distributed databases èMichael p Data èNatalia Zolakar (WBS 1. 8/1. 7) storage and date import/export issues Ratnikova: 100% UF (WBS 1. 2/1. 7/1. 8) p Will start October 30; Code librarian, code distribution and documentation èJoe Kaiser is the “old man” on CMS UF. He started Aug, 2000. Personnel are still getting up to speed. We are leveraging from FNAL employees already working on CMS (CAS engineers, Guest Scientists, other CD departments) October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 28

Current Status of FNAL User Facility Current Hardware Resources : CPU: èSUN E 4500 Current Status of FNAL User Facility Current Hardware Resources : CPU: èSUN E 4500 with 8× 400 MHz CPU’s (CMSUN 1) è 3 4 -CPU 650 MHz Linux Systems (Wonder, Gallo, Velveeta) è 2 2 -CPU 500 MHz Linux Systems (Boursin, Gamay) Disk: è 125 GB local disk space è 1 TB RAID on CMSUN 1 è¼ TB RAID on each Wonder, Gallo, Velveeta Storage è 3 TB in HPSS tape library è 20 TB in Enstore tape library New Systems è 40 dual 750 MHz CPU linux farm (popcrn 01 – 40) èIn burn-in phase – will be released to users end of October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 29

Current Status of FNAL User Facility CMS software support The full CMS software environment Current Status of FNAL User Facility CMS software support The full CMS software environment is supported on Sun and linux: (although SUN support lags a bit behind) èCMSIM 120, ORCA 4_3, SCRAM, IGUANA, GEANT 4 CMS user support Software Developers èSupport s/w developers psupport full CMS software installation at FNAL poperate a standard code-build facility for code development + test èsupport other US institutions to fulfill their software development responsibilities October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 30

Current US-CMS User Support Monte Carlo Studies è Currently supporting US CMS groups studying Current US-CMS User Support Monte Carlo Studies è Currently supporting US CMS groups studying Higher Level Triggers p Provide dedicated effort to make a system for MC production p Provide data storage and transport across US CMS institutions p Provide data analysis resources èPhysics studies èDetector Monte Carlo studies Test Beam Studies èSupport requests Miscellaneous èHold biweekly meetings over VRVS to discuss support issues with users p Every other Tuesday, 1 pm CST, Saturn virtual room (alternates with US-CMS physics meeting) October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 31

Training Plans An important part of the User Facility is training for both users Training Plans An important part of the User Facility is training for both users and staff. Fermilab offers C++ and OOAD classes regularly. All CMS people are free to register for such classes. In addition, given enough interest we could organize a special C++ or OOAD class for CMS. In addition we just had a CMS specific training workshop for beginners and also design discussions for the experts October 12 -14. (http: //home. fnal. gov/~cmsdaq/tutorial. html) è ~ 60 Attendees, 45 from outside institutions. About 40 were beginners, the other 20 closer to expert. è Experts from CERN came to give the tutorials. All material was installed and running on Fermilab CMS machines, where people actually ran the tutorial hands on. è Good opportunity to meet CMS colleagues and to discuss s/w design details as well. è We plan to do this as often as there is interest, hopefully twice a year or so. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 32

Summary A new “bottoms up” hardware cost has been made for the User Facility Summary A new “bottoms up” hardware cost has been made for the User Facility Project. We have a fledgeling costbook to justify these estimates. The hardware requirements are coming out of CMS through the Hoffman review panel. We are tracking these estimates closely. A new resource loaded, bottoms-up estimate for personnel requirements has been made for the User Facility Project. This estimate agrees with the old “top-down” approach to within 5% or so. This talk has just been a brief overview of our current project status, but tomorrow some of you will have the pleasure of going through the cost estimates and WBS in detail. October 23, 2000 Vivian O’Dell, US CMS User Facilities Project, USCMS S&C Oversight Panel Review 33