LHC Physics or How LHC experiments will use

LHC Physics or How LHC experiments will use OSG resources OSG Consortium Meeting 08 / 21 / 06 Oliver Gutsche USCMS / Fermilab

Particle Physics Probe innermost structure and explain it from first principles 08/21/06 Oliver Gutsche - LHC Physics 2

Standard Model Current description of matter: 12 elementary particles and their anti-particles 4 force particles carrying the 3 dominating forces (at scales investigated) 08/21/06 Oliver Gutsche - LHC Physics 3

LHC, the discovery machine What can the LHC add to the current picture: Higgs: explain the origin of mass Super-Symmetry: beyond the standard model 08/21/06 Oliver Gutsche - LHC Physics 4

LHC Large Hadron Collider Proton-Proton collisions Beam energy: 7 Te. V Circumference: 27 km 08/21/06 Bunch crossing rate at interaction regions: 40 MHz Oliver Gutsche - LHC Physics 5

Outline Example for Higgs discovery channel Analysis and requirements Computing CMS Metrics CMS computing model The role of the T 2 centers OSG contribution User use case Services Challenges and current status Summary & Outlook 08/21/06 Oliver Gutsche - LHC Physics Apologies to the other 3 experiments, following talk mainly concentrates on CMS 6

Example Signal: (*)→ 4μ H→ZZ Signal Reconstruction strategy: Reconstruct 4 μ (2 μ+, 2 μ-) combine each 2 μ to Z combine 2 Z to H 08/21/06 Oliver Gutsche - LHC Physics 7

Event display 08/21/06 Oliver Gutsche - LHC Physics 8

Backgrounds and Analysis Strategy. Signal overlaid by background events which produce similar signature in detector 3 main background Analysis Strategy: Record data with detector tt Zbb 08/21/06 Simulate background and signal events ZZ Extract signal from recorded data using the background simulation Oliver Gutsche - LHC Physics 9

Data Taking High Level Trigger Reconstructio n primary datasets MC production Data taking vs. MC production Generatio n and Simulation Reconstructio n MC datasets 08/21/06 Oliver Gutsche - LHC Physics 10

Data tiers data tier description RAW digitized detector output Sim. Event generated and simulated event information RECO reconstructed information AOD extract from RECO, most used analysis quantities RECOSim. Event AOD extract from RECO, most used analysis quantities FEVT (full event) Sim. Event AO D 08/21/06 RECO-Sim. Event RECO RAW Sim. FEVT (full simulated event) Oliver Gutsche - LHC Physics 11

Disk requirements Detector output rate after trigger: 150 Hz for all running conditions (low luminosity, high luminosity) Extrapolated beam time: year beam time number of events 2007 106 s 1. 5 108 2008 107 s 1. 5 109 data_tier total size in 2008 RAW 1. 50 MB 0. 21 PB 2. 10 PB Reco 0. 25 MB 0. 03 PB 0. 35 PB AOD 0. 05 MB 0. 01 PB 0. 07 PB 1. 80 MB 0. 25 PB 2. 51 PB data_tier Events from MC production: total size in 2007 total Events from data taking: event size total size in 2007 total size in 2008 0. 28 PB 2. 79 PB 2. 00 MB Sim. Event Reco. Sim. Event 0. 40 MB 0. 06 PB 0. 56 PB AOD 0. 10 MB 0. 01 PB 0. 14 PB total 2. 50 MB 0. 35 PB 3. 49 PB year 2007 Total: 08/21/06 total data volume 0. 60 PB 2008 6. 01 PB Oliver Gutsche - LHC Physics 12

CPU requirements Reconstruction and MC production Assume Core in 2007: 2000 SI 2 K (optimistic) Single Core Pentium IV 3 GHz ≈ 1300 SI 2 K Time to reconstruct event Analysis Single Analysis has to access one primary dataset and the MC samples Assume: Analysis needs access to AOD Every 3 days 78 k. SI 2 K s/ev On demand reconstruction at 150 Hz 11. 7 MSI 2 K ≈ 5850 Core’s Selection has to be finished at least after 3 days Analysis needs access to RECO Every 7 days Time to simulate and reconstruct event 234 k. SI 2 K s/ev => 1 Core ≈ 270, 000 ev/year 08/21/06 Analysis has to be finished at least after 7 days Selection time: 0. 25 k. SI 2 K s/ev Analysis time: 0. 25 k. SI 2 K s/ev Oliver Gutsche - LHC Physics 13

CMS Grid Tier Structure T 2 T 2 T 2 T 1: USA T 2 T 1: Italy T 1: France T 2 T 0 T 1: Germany T 2 T 1: Spain T 2 T 2 T 1: UK T 1: Taiwan T 2 T 2 08/21/06 T 2 T 2 Oliver Gutsche - LHC Physics T 2 14

Data flow 7 T 1 T 0 distributes RAW and reconstructed data to T 1’s (subset of the primary datasets, full AOD copy) T 2’s are associated to specific T 1 which provides support and distributes data (simulated MC is transferred back to associated T 1) 08/21/06 ➡ 25 T 2 substantial computing resources are provided by the T 1’s and T 2’s CMS-CAF performs latency critical activities like detector problem diagnostic, trigger performance service, derivation of calibration and alignment data Oliver Gutsche - LHC Physics 15

US contribution to CMS Tier structure U. S. contribution to CMS tier structure T 1 at FNAL 7 associated T 2 sites (OSG) Wisconsin T 2 Nebraska T 1 T 2 FNAL MIT T 2 Purdue CALTECH T 2 San Diego Florida T 2 08/21/06 Oliver Gutsche - LHC Physics 16

DBS: Data Bookkeeping System CRAB Prod. Agent Ph. EDEx CMS Computing Infrastructure Core official catalog of available datasets and MC samples organizes datasets and samples in fileblock containing files GRID Middleware Site Resources batchsystem CPU 08/21/06 catalog of which storage elements contain specific datasets and MC samples DLS DBS DLS: Data Location Service TFC: Trivial File Catalog files are stored in a CMS specific namespace which is reproduced at the storage elements at every site TFC Access to files in the CMS namespace is resolved by the CMS applications automatically prepending site specific locations and access protocol using a local site configuration Disk Oliver Gutsche - LHC Physics 17

CRAB Prod. Agent Ph. EDEx CMS Computing Infrastructure Computing Systems Ph. EDEX Transport any dataset or samples registered in DBS between sites DLS DBS GRID Middleware Site Resources batchsystem 08/21/06 Produce MC samples in centralized way User Analysis Tool (CRAB) TFC CPU MC production system (Prod. Agent) Disk enable user to execute his analysis code on any sample registered in DBS/DLS Oliver Gutsche - LHC Physics 18

Site requirements CMS T 2 site CPU: 1 MSI 2 K DISK: 200 TB Network: 2. 5 -10 Gbps Opportunistic usage (OSG) OSG middleware stack Useable for MC production Computing Element (CE) Storage Element (SE) OSG middleware stack Batch system OSG sites: Condor and PBS Outbound connectivity of Mass Storage system workernodes (stage-in and out) OSG sites: d. Cache (tapeless) CMS software installed by central instance (OSG sites: $OSG_APP) including TFC setup 08/21/06 Computing Element (CE) CMS software installed by central instance (in $OSG_APP) Oliver Gutsche - LHC Physics 19

User analysis Task Discovery of the Higgs in channel H→ZZ(*)→ 4μ Approach: use recorded and reconstructed data from detector OLI use produced MC samples reconstruct 4μ signature and extract Higgs mass 08/21/06 Oliver Gutsche - LHC Physics 20

MC samples: ideal MC workflow Produced by MC production system at T 2 level Archived at T 1 which stores corresponding datasets MC production requests Initiated by the physics groups Samples are probably more general than needed Physics groups and users Prepare skims of the MC samples at T 1’s for specific physics purposes Complete skims are transported to dedicated T 2’s. 08/21/06 User analyzes MC samples by processing skims at Oliver Gutsche - LHC Physics T 2’s 21

ideal Data workflow Data Recorded by the detector Triggered by the HLT Reconstructed at T 0 Split into primary datasets Distributed between T 1 centers Most analysis only needs to access one of the primary samples AOD is sufficient for 90% of the analysis data taking hasn’t started, for now just a event display of a cosmic muon Physics groups and users Skim AOD of primary samples at T 1’s Complete skims are transported to dedicated T 2’s. User analyzes data samples by processing AOD skims at T 2’s 08/21/06 Oliver Gutsche - LHC Physics 22

Special tasks Alignment/Calibration In general, alignment/calibration is calculated at T 0 or T 1 for immediate usage during reconstruction Parts of RAW data samples can in addition be transported to dedicated T 2 to calculate improved alignment/calibration constants Alignment / Calibration Reprocessing of data Data samples are planned to be rereconstructed with improved alignment/calibration and reconstruction algorithms 3 times a year on T 0 and T 1 level Reprocessin g Reprocessing requires re-skimming of samples 08/21/06 Oliver Gutsche - LHC Physics 23

Startup scenario and During startup of LHC, no experience with the detector its output will exist First calibration/alignment calculations will be not sufficient for analysis AOD RAW data samples will be transported to T 2’s to: Understand the detector Improve the reconstruction algorithms, calibration and alignment Extract first physics messages T 1 resources Used more for analysis than skimming and rereconstruction (less resources needed by default operations). 08/21/06 Re-reprocessing will be more frequent but short running time hasn’t produced large amounts of data yet Oliver Gutsche - LHC Physics RAW 24

Service Challenges To prepare for the startup, the experiments exercise their systems in dedicated service challenges CMS currently runs “Service Challenge 4” (SC 4) which consists of the exercise of: the dataset transport system Ph. EDEx with emphasis on all possible transport endpoint combinations (T 0, T 1, T 2) the MC production system by running ~12, 500 MC production jobs per day on LCG and OSG resources and to produce MC samples for the upcoming challenge the analysis infrastructure by running ~12, 500 analysis jobs per day on samples distributed to every center CMS’ next challenge is CSA 2006 in the fall which exercises the full workflow feeding MC samples to the HLT and reconstruction at T 0 centers Reprocessing, skimming at T 1 centers Analysis and MC simulation at T 2 centers Sample transportation between the individual centers 08/21/06 Oliver Gutsche - LHC Physics 25

First MC production run: OSG summary First MC production run: ~2 weeks of August Total: 45 Mio. events, OSG (incl. FNAL): 17 Mio. events More details in Ajit Mohapatra’s talk on Wednesday 08/21/06 Oliver Gutsche - LHC Physics 26

Analysis infrastructure: grid-wide performance Overall Job statistics GRID Success Statistics 08/21/06 Job Success Statistics Automated systems (Job. Robots) achieve gridwide performance close to goal of 12, 5000 jobs per day scale at ~10, 000 jobs per day for 5 days and improving 27 Oliver Gutsche - LHC Physics

Analysis infrastructure: OSG-wide performance Overall Job statistics GRID Success Statistics Job Success Statistics Scale at ~3, 000 jobs per day for 5 days OSG sites contribute large percentage of overall analysis performance 08/21/06 Oliver Gutsche - LHC Physics 28

Analysis infrastructure: OSG site performance Overall Job statistics GRID Success Statistics Job Success Statistics Example: Nebraska: scale between 450 and 850 jobs per day high success rates Other OSG sites show similar performance 08/21/06 Oliver Gutsche - LHC Physics 29

Summary & Outlook Computing for the LHC experiments is large and globally organized LHC computing marks the final change in high energy physics from the host laboratory centered analysis of data to a global approach GRID resources and OSG resources in particular will be used efficiently to perform standard production and analysis as well as special tasks Analysis will be a challenge, not only from the physics point of view but also from the computational requirements and specialties 08/21/06 Oliver Gutsche - LHC Physics 30

In the end. . . We hope to make many discoveries in particle physics with LHC data using OSG resources 08/21/06 Oliver Gutsche - LHC Physics 31

The end 08/21/06 Oliver Gutsche - LHC Physics 32