SAM Tevatron Experiments Using the Grid Rick St

Скачать презентацию SAM Tevatron Experiments Using the Grid Rick St

804f0b3c02a964e91ab99ad31d1b2c46.ppt

Количество слайдов: 102

SAM: Tevatron Experiments Using the Grid Rick St. Denis, University of Glasgow • CDF and D 0 Need the Grid – Requirements, the CAF and SAM – Grid from the User Perspective • Grid to Meet the Need – How SAM works – SAM usage by D 0 and CDF • Near Future: SAMGrid 11 March 2004 Getting Ready for the Grid

Spokespersons’ Requirements for CDF Maximize physics output @ low Lumi –L 3 output rate: 80 -> 360 Hz by 06 Reviews: Director’s (technically), CDF needs the Grid International Finance Committee (fiscally) FNAL PAC (for its physics merit) 50% computing outside FNAL 11 March 2004 Getting Ready for the Grid

Scale of CDF Requirements THz FY 04 3. 7 %offsite CPU Speed 25% 3 GHz #duals FY 05 9. 0 50% 5 GHz +360 FY 06 16. 5 50% 8 GHz +220 150 6 -7 sites, 100 Duals each, by 2006 + 700 @FNAL 11 March 2004 Getting Ready for the Grid

CDF Computing Model • Develop Analysis on desktop – Access to all CDF data from Exists Now anywhere • Large scale processing on batch clusters – Submission from anywhere Implemented Now with – interactive tools: ls, top, head/tail/cat CAF (not Grid standard) – Output to scratch space or desktop 11 March 2004 Getting Ready for the Grid

Central Analysis Facility • CAF is a pile of PC’s with a pile of disks. (1200 processors and 100 TB) • This can be implemented anywhere as d. CAF: Decentralized CAF. • Output of jobs can go to desktop or a scratch area • Need a password for this: authentication (kerberos). 11 March 2004 Getting Ready for the Grid

Sequential Access through Metadata • Metadata: SAM allows groups of files to be identified into datasets using attributes (metadata) such as production pass version or top quark mass to associate them. • File Retrieval: SAM moves files to users as they request them. • File Storage: SAM allows output files to be stored with new metadata. 11 March 2004 Getting Ready for the Grid

Metadata [sam@nglas 08 ~]$ sam get metadata --file=Bs_conc_4 o 5_3. root File Type: SAMMC Data File totalevents = 7290 File Name: Bs_conc_4 o 5_3. root Work Group: cdf File ID: 2494282 html = http: //www. pd. infn. it/~lucchesi Node Name: cdfsam. cnaf. infn. it File Size: 530926740 [B] File Start Time: 01/29/2004 16: 00 File End Time: 01/29/2004 17: 00 Application Family: generator dataset = Bs. MC-lucchesi_test Application Version: 1. 00 Description: Bs. Dspi_phipi MONTE CARLO Dataset 4 o 5 part 3 11 March 2004 Run Number: 167634 Getting Ready for the Grid

Use Cases • User Level MC Production – All Users have access – No data on site -> write to tape at FNAL • User Level SAM provides this Data Access – All users have access – Selected samples automaticaly copied on site 11 March 2004 Getting Ready for the Grid

Functionality • User selects a place to run, saying what dataset they will use • System checks they can do this (privileges) • User access to data at any place • User output is stored on any disk or back to tape at FNAL and results are made available for transfer to any site for others to analyse. 11 March 2004 Getting Ready for the Grid

User Perspective CAF Gui/CLI Uses SAM Analysis program Grid Italy 11 March 2004 Toronto Korea Only. Grid. Lab Outside Fermilab Taiwan Fermi. CAF Getting Ready for the Grid UK

Meeting the Needs • • SAM: How it works Progress in SAM CDFGrid. Workshop: “Nerd’s Paradise” D 0 and CDF Usage 11 March 2004 Getting Ready for the Grid

Fcdfdata 016 Disk/Cache FSS (Deamon) (fss) Stager Station Stager Daemon (stagerng) central-analysis Daemon (smaster) Stager Daemon (stagerng) 11 March 2004 Stager Daemon (stagerng) Getting Ready for the Grid Disk/Cache

A Farm: Station with Stagers and Caches Cache Cache Node 1 Node 2 Node 3 Node 4 Node 5 Stager Station Stager stagerng smaster Stager stagerng 11 March 2004 Getting Ready for the Grid

What can 20 duals and 6 TB do? Stream Events Days Input Size Top, W/Z 20. 5 M 10. 3 Hadronic B 156 M and charm 78. 3 4. 5 TB 34. 2 TB Need to transfer 0. 6 GB/min or 1 TB/Day 11 March 2004 Getting Ready for the Grid

<fcdfdata 016> 11 March 2004 fcdfdata 016 Getting Ready for the Grid Disks/Cache 11 March 2004 fcdfdata 016 Getting Ready for the Grid Disks/Cache

<fcdfdata 016> fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Disks/Cache

<fcdfdata 016> fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016>sam submit --script=userscript --group=groupname --cpu-per-event= --defname= fcdfdata 016 Station central-analysis smaster 11 March sam submit --script=userscript --group=groupname --cpu-per-event= --defname= fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> >>>>>> Starting project with the Station Master contacted, result: Started project 49008 >>>>>> Starting project with the Station Master contacted, result: Started project 49008 (49008_sam_) for group test Waiting for the project to initialize. . . fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> fcdfdata 016 Callback from server: 'OK|Project is ready' Station central-analysis smaster Project fcdfdata 016 Callback from server: 'OK|Project is ready' Station central-analysis smaster Project pmaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> >>>>>> Submitting the job to the batch system. fcdfdata 016 Station central-analysis >>>>>> Submitting the job to the batch system. fcdfdata 016 Station central-analysis smaster Project pmaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 PSUSP central-analysis smaster Project pmaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 PSUSP central-analysis Disks/Cache Stager stagerng smaster Optimizer Project pmaster 11 March 2004 Getting Ready for the Grid

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Stager Station Batch Job <52554> is submitted to queue . fcdfdata 016 Stager Station Batch (LSF) 52554 PSUSP Disks/Cache stagerng central-analysis smaster eworker Project eworker pmaster 11 March 2004 Getting Ready for the Grid eworker

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Stager Station Batch Job <52554> is submitted to queue . fcdfdata 016 Stager Station Batch (LSF) 52554 PSUSP Disks/Cache stagerng central-analysis smaster eworker Project encp eworker pmaster encp 11 March 2004 Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Stager Station Batch Job <52554> is submitted to queue . fcdfdata 016 Stager Station Batch (LSF) 52554 PSUSP Disks/Cache Enstore stagerng central-analysis smaster eworker Project encp eworker pmaster encp 11 March 2004 Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 PSUSP central-analysis Disks/Cache Stager Enstore stagerng smaster eworker Project pmaster 11 March 2004 Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager Enstore stagerng smaster eworker Project pmaster 11 March 2004 Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager stagerng smaster samscript. sh userscript Enstore eworker Project pmaster 11 March 2004 Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 Enstore consumer Getting Ready for the Grid encp

SAMManager: sam Getting next input file. . . SAMManager: sam Project master will call back. 11 March 2004 Getting Ready for the Grid

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 Enstore consumer Getting Ready for the Grid encp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis smaster samscript. sh userscript Project pmaster 11 March 2004 consumer Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Disks/Cache rm Station Job <52554> is submitted to queue . fcdfdata 016 Disks/Cache rm Station Batch (LSF) 52554 RUN central-analysis smaster samscript. sh userscript Project pmaster 11 March 2004 consumer Getting Ready for the Grid Stager stagerng rm

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis smaster samscript. sh userscript Project pmaster 11 March 2004 consumer Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Stager stagerng smaster samscript. sh userscript Optimizer Project pmaster 11 March 2004 Disks/Cache consumer Getting Ready for the Grid

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Stager stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 Disks/Cache consumer Getting Ready for the Grid eworker

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 consumer rcp Getting Ready for the Grid rcp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager Other Cache stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 consumer rcp Getting Ready for the Grid rcp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis Disks/Cache Stager stagerng smaster samscript. sh userscript eworker Project pmaster 11 March 2004 consumer rcp Getting Ready for the Grid rcp

<fcdfdata 016> Job <52554> is submitted to queue <sam_lo>. fcdfdata 016 Station Batch (LSF) Job <52554> is submitted to queue . fcdfdata 016 Station Batch (LSF) 52554 RUN central-analysis smaster samscript. sh userscript Project pmaster 11 March 2004 consumer Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> fcdfdata 016 Station Batch (LSF) central-analysis smaster Project pmaster 11 March 2004 fcdfdata 016 Station Batch (LSF) central-analysis smaster Project pmaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016> Batch (LSF) fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready Batch (LSF) fcdfdata 016 Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Disks/Cache Stager stagerng

<fcdfdata 016>sam submit…. fcdfdata 016 Disks/Cache <fcdfdata 016>sam submit…. <fcdfdata 016>sam run project… Batch sam submit…. fcdfdata 016 Disks/Cache sam submit…. sam run project… Batch (LSF) Station central-analysis smaster 11 March 2004 Getting Ready for the Grid Stager stagerng

<fcdfdata 016> fcdfdata 016 52668 <user 1> RUN 52675 <user 2> RUN 52756 <user fcdfdata 016 52668 RUN 52675 RUN 52756 PSUSP samscript. sh userscript Enstore Other Cache stagerng central-analysis eworker smaster eworker Project pmaster consumer 11 March 2004 Stager Station Batch (LSF) Disks/Cache Project pmaster consumer encp Project pmaster Getting Ready for the Grid rcp

Storing Files Getting things to tape from Glasgow 11 March 2004 Getting Ready for the Grid

<fcdfdata 016> fcdfdata 016 Disks Stager FSS Central-analysis fss 11 March 2004 Getting Ready fcdfdata 016 Disks Stager FSS Central-analysis fss 11 March 2004 Getting Ready for the Grid stagerng

<fcdfdata 016>sam store descrip. py --source=<file loc> [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager sam store descrip. py --source= [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager FSS Central-analysis fss 11 March 2004 Getting Ready for the Grid stagerng

<fcdfdata 016>sam store descrip. py --source=<file loc> [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager sam store descrip. py --source= [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager FSS Descrip. py Metadata Info about file Central-analysis fss Sam checks info, checks location, 11 March 2004 Getting Ready for the Grid stagerng

<fcdfdata 016>sam store descrip. py --source=<file loc> [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager sam store descrip. py --source= [--dest=/pnfs…. . ] fcdfdata 016 Disks Stager FSS stagerng Central-analysis fss eworker encp, rcp, bbftp 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Disk Tmp Disk fcdfdata 016 Stager Fss From Really Far Away central-analysis Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Disk Tmp Disk fcdfdata 016 Stager Fss Routing: fcdfdata 016 sam store central-analysis enstore Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Disk Tmp Disk Stager Fss fcdfdata 016 Fss Routing: fcdfdata 016 central-analysis eworker bbftp fcdfdata 016 Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Disk Tmp Disk Stager Fss fcdfdata 016 Fss From really Far away central-analysis eworker bbftp fcdfdata 016 Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Disk Tmp Disk fcdfdata 016 Stager Fss From really Far away central-analysis Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Disk Tmp Disk fcdfdata 016 Stager Fss From really Far away central-analysis eworker encp Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Stager Disk Tmp Disk rm fcdfdata 016 Stager Fss From really Far away central-analysis Enstore 11 March 2004 Getting Ready for the Grid

Node from Really Far Away Fss Disk Tmp Disk fcdfdata 016 Stager Fss From really Far away central-analysis Enstore 11 March 2004 Getting Ready for the Grid

D 0 Sam D 0 relies entirely on SAM for analysis 11 March 2004 Getting Ready for the Grid

D 0 11 March 2004 Getting Ready for the Grid

D 0 Files 4000 -8000 Files/Day 11 March 2004 Getting Ready for the Grid

D 0 Data Volume 1 TB-3 TB/day 11 March 2004 Getting Ready for the Grid

D 0 Files Per Month By Year 1999 2000 2001 100, 000 files Run II Start 11 March 2004 Getting Ready for the Grid 2002 2003

D 0 Total Files 2. 5 Million Files Served 11 March 2004 Getting Ready for the Grid

D 0 Data Per Month By Year 1999 2000 2001 50 TB per month Run II Start 11 March 2004 Getting Ready for the Grid 2002 2003

D 0 Total Data Moved 700 TB moved 11 March 2004 Getting Ready for the Grid

Progress in SAM: CDF • All 800, 000 CDF data files are in SAM • Sam is in beta testing on the CDF CAF (1200 cpus): passed 20 TB/Day delivery • Karlsruhe uses SAM routinely • Minos uses SAM for its Data Handling • Steve Mrenna (Phenomenology) depositing ALPGEN files in SAM for common CDF/D 0 use. 11 March 2004 Getting Ready for the Grid

Florida workshop: • 11 installations in about 2 hours. Integrated with d. CAF in 2 cases in 2 days. Now 20! • 3 in Asia, 4 in Europe • 6 sites committed to summer 2004 usage of their facilities for all of CDF (mostly MC) • Sam installation now: initsam cdf • Follow-up on April 1. • Each site has a local user support person to reduce load on core development team. • Generally: Security ate 80% of the effort! 11 March 2004 Getting Ready for the Grid

CDF 11 March 2004 Getting Ready for the Grid

Florida Workshop: After 2 Days 11 March 2004 Getting Ready for the Grid

2 TB/Day: Karlsruhe 11 March 2004 Getting Ready for the Grid

CDF Dcache on CAF ALL CDF on CAF reads 25 TB/Day Non. Grid Running 11 March 2004 Getting Ready for the Grid

CDF Files in a Month Karlsruhe: 1500 files/Day 11 March 2004 Getting Ready for the Grid

CDF Events Transfer in a Month Karlsruhe: 5 -10 M Evt/Day 11 March 2004 Getting Ready for the Grid

All CDF Files Moved by SAM 2002 2003 300 K Files D 0: 2. 5 M files 11 March 2004 Getting Ready for the Grid

Total CDF Data Moved 2002 2003 200 TB D 0: 700 TB 11 March 2004 Getting Ready for the Grid

Advantage of Local Processing • Karlsruhe processes 2 TB/day. Rest of CDF on Central Cluster processes 25 TB/day. (450 processors, 8 experiments, 10/13 TB disk filled. ) • 5 users actively at Karlsruhe. Make ntuple for bottom and top physics for 15 people. • 100 users active for rest of CDF: • They pin the datasets of interest; copy new ones automatically. 11 March 2004 Getting Ready for the Grid

In the near term future: JIM Adding Grid Standard Tools 11 March 2004 Getting Ready for the Grid

CDF Grid Strategy • 25% of CDF Computing from external resources. All CDF computing on CDF Grid by April 15: Utilize resources fully controlled by CDF: Kerberos/fbsng: d. CAF + SAM • October 15, 2004: JIM to capture shared resources • June 2005: 50% of Computing resources external 11 March 2004 Getting Ready for the Grid

Anywhere @ each site Desktop Simple JIM Private LAN Globus GK CAF Submitter SAM Station @regional centers Condor Submitter WN Private LAN d. Cache @FNAL SAM DB Condor Matchmaker 11 March 2004 Getting Ready for the Grid June 2004 testing June 2005 required

Detailed JIM User Interface Flow of: job User Interface Submission Global Job Queue data meta-data User Interface Submission Resource Selector Match Making Global DH Services Info Gatherer SAM Naming Server Info Collector Grid Client SAM Log Server Resource Optimizer MSS Cluster Data Handling Grid Gateway SAM DB Server Site RC Local Job Handling SAM Station (+other servs) Cache SAM Stager(s) Local Job Handler (CAF, D 0 MC, BS, . . . ) JIM Advertise Dist. FS 11 AAA March 2004 Worker Nodes Meta. Data Catalog Bookkeeping Service Info Manager MDS Info Providers Web Serv Grid Monitoring XML DB server Site Conf. Glob/Loc JID map Getting Ready for the Grid. . . User Tools Site

Conclusions • CDF has embraced the need for the Grid to achieve its physics mission • SAM is working for D 0 and growing in CDF 11 March 2004 Getting Ready for the Grid