Summary Distributed Data Analysis Track coordinators F Rademakers

Summary Distributed Data Analysis Track coordinators F. Rademakers, S. Dasu, V. Innocente G. Ganis AA meeting, Feb 22, 2005 CHEP 06, 17 Feb 2006 CHEP 06 TIFR, Mumbai Most of the slides are from Fons’ plenary talk Fons Rademakers

What Was This Track All About? 23 talks Analysis Systems Grid Simulations Job. Mon Ganga Bb. K Submission Systems BOSS Data Access Systems AJAX Analysis CRAB Job. Mon PANDA DIRAC Go 4 DIAL SAM Prod. Sys BOSS Dash. Board BOSS PROOF Monitoring Systems Mona. Lisa Bookkeeping Systems ARDA xrootd Miscellaneous CHEP 06, 17 Feb 2006 2 Fons Rademakers

Data Analysis Systems u The different analysis systems presented, categorized by experiment: ALICE PROOF u u ATLAS DIAL GANGA CMS CRAB PROOF LHCb GANGA All systems support, or plan to support, parallelism Except for PROOF all systems achieve parallelism via job splitting and serial batch submission (job level parallelism) CHEP 06, 17 Feb 2006 3 Fons Rademakers

Classical Parallel Data Analysis catalog files Batch farm Storage queues query data file splitting jobs my. Ana. C merging final analysis submit manager outputs § “Static” use of resources § Jobs frozen, 1 job / CPU § “Manual” splitting, merging § Limited monitoring (end of single job) § Possible large tail effects CHEP 06, 17 Feb 2006 4 Fons Rademakers

Interactive Parallel Data Analysis catalog files Interactive farm Storage scheduler query: data file list, my. Ana. C MASTER feedbacks final outputs (merged) § Farm perceived as extension of local PC § More dynamic use of resources § Automated splitting and merging § Real time feedback § Much better control of tail effects CHEP 06, 17 Feb 2006 5 Fons Rademakers

I. Gonzales CHEP 06, 17 Feb 2006 6 Fons Rademakers

Batch-oriented D. A. Systems u GANGA: cooperation between LHCb and ATLAS n n n u Designed for analysis on the Grid (T 1) Backend type in job definition: local, local batch, grid (via DIRAC) Facilitate bookkeeping, job tracking DIAL (ATLAS: Distributed Interactive analysis on Large Datasets) n n n Web service framework for common interface to large range of batch and workload management systems Insulate users from splitting, submission, merging and error recovery Performance for a reference dataset (see later) 1872 files, 100 evts each, replicated 6 times l Reference atlasdev analysis (transformation) l u CRAB (CMS) n n CHEP 06, 17 Feb 2006 Python tool to facilitate creation of a large number of user analysis job Efficient access to data hiding middleware complications Manages submission, tracking, monitoring and harvesting Used for physics TDR and SC 3 (WLCG, OSG) 7 Fons Rademakers

GANGA U. Egede CHEP 06, 17 Feb 2006 8 Fons Rademakers

GANGA U. Egede CHEP 06, 17 Feb 2006 9 Fons Rademakers

DIAL 15 x CHEP 06, 17 Feb 2006 D. Adams 10 Fons Rademakers

CRAB M. Corvo CHEP 06, 17 Feb 2006 11 Fons Rademakers

Submission Systems u The different submission systems, categorized by experiment: ALICE Ali. En (not presented) u ATLAS Prod. Sys Pan. DA CMS BOSS LHCb DIRAC These systems are the DDA launch vehicles for the Grid based batch analysis solutions CHEP 06, 17 Feb 2006 12 Fons Rademakers

Submission systems u ATLAS n Prod. Sys: provide seamless access to all ATLAS grid resources l emphasis on batch model u interactive solutions difficult to realize on top of the current middleware layer n Pan. DA: Production and Distributed Analysis system (OSG) Job/Executor interface, Task buffer, Brokerage, Dispatcher, Data Service, Job scheduler, Logging / Monitoring system l Very recent project developed by the ATLAS U. S. team l u CMS n BOSS: Batch Object Submission System l u tool for batch job submission, real time monitoring and book keeping LHCb n DIRAC: Workload and Data management system Pull scheduling paradigm via pilot agent technology l Multi-threaded mode show to reduce job start times l u Pilot agents request several jobs from the same user and run two jobs in parallel CHEP 06, 17 Feb 2006 13 Fons Rademakers

Prod. Sys D. Liko CHEP 06, 17 Feb 2006 14 Fons Rademakers

BOSS Workflow Wrapper BOSS Scheduler farm node boss submit boss query boss kill u n n u farm node User specifies job - parameters including: n u BOSS DB Executable name. Executable type - turn on customized monitoring. Output files to retrieve (for sites without shared file system and grid). User tells Boss to submit jobs specifying scheduler i. e. PBS, LSF, SGE, Condor, LCG, GLite etc. . Job consists of job wrapper, Real time monitoring service and users executable. CHEP 06, 17 Feb 2006 15 Fons Rademakers From Evolution of BOSS by Wakefield [240]

New data flow BOSS S. Wakefield CHEP 06, 17 Feb 2006 16 Fons Rademakers

DIRAC S. K. Patterson CHEP 06, 17 Feb 2006 17 Fons Rademakers

DIRAC S. K. Patterson CHEP 06, 17 Feb 2006 18 Fons Rademakers

Data Access Systems u The different data access systems that were presented: n SAM (Sequential data Access via Meta-data) Used by CDF in its CAF environment l Generic tool also providing monitoring, bookkeeping, automatic error recovery l n xrootd server Used by Ba. Bar (all sites), ALICE, STAR (larger deployment) l Linear scaling relative to load allows deterministic sizing of servers l n xrootd client l CHEP 06, 17 Feb 2006 Reduce latencies using asynchronous read ahead, client side caching and asynchronous opens 19 Fons Rademakers

XROOTD A. Hanushevsky CHEP 06, 17 Feb 2006 20 Fons Rademakers

XROOTD client CHEP 06, 17 Feb 2006 F. Furano 21 Fons Rademakers

XROOTD client simulation F. Furano CHEP 06, 17 Feb 2006 22 Fons Rademakers

Summary u Several activities on Distributed Data analysis tools n n n u Several grid independent access layers n n u GANGA, DIAL, CRAB, PROOF, … Still development phase “Tools not yet been to the expected large number of final analysis users”(*) DIRAC, BOSS, ALi. En, Pan. DA, … “Gap between the grid middleware capabilities and user needs, especially data location, placement and bookkeeping services” (*) Low latency storage access may boost analysis efficiency (*) from Conveners’ observations CHEP 06, 17 Feb 2006 23 Fons Rademakers