Applications Area Plans for Phase 2 AA Meeting CERN

Applications Area Plans for Phase 2 AA Meeting/CERN 5 October 2005 Pere Mato/CERN 05/10/05 LCG Applications Area Plans

Outline u The planning exercise u Update on SEAL and ROOT merge u Projects, Milestones and Resources 05/10/05 LCG Applications Area Plans 2

AA Program of Work Phase 2 u Last week the AA Planning Document was approved by the PEB – http: //lcgapp. cern. ch/project/docs/AA_Program_of_Work_Phase 2_final. pdf u u u This ends the planning exercise that started in May/June The plan is consistent with the change proposals presented at the AA internal review in April Defined about 80 level-2 milestones to follow progress of the projects Accounted resources being invested in each project Most problematic area in the plan has been to reach an agreement on the SEAL and ROOT project merge – Compromise found and described later in the talk 05/10/05 LCG Applications Area Plans 3

Scope and Requirements u The overall scope for AA is the same as for LCG Phase I – establishment of the basic environment for software development, documentation, distribution and support – development of general-purpose scientific libraries, C++ foundation libraries, and other standard libraries – development of tools for storing, managing and accessing data handled by physics applications – adaptation and validation of common frameworks and toolkits provided by projects of broader scope than LHC, such as Pythia, Geant 4 and Fluka. u The main requirements are also basically the same as for Phase I 05/10/05 LCG Applications Area Plans 4

Principal Architecture Requirements u u u Long lifetime: support technology evolution C++ today; support language evolution Seamless distributed operation and usability off-network Component modularity, public interfaces Interchangeability of implementations Integration into coherent framework and experiment software Design for end-user’s convenience more than the developer’s Re-use existing implementations Software quality at least as good as any LHC experiment Meet performance, quality requirements of trigger/DAQ software Platforms: Linux/gcc, Windows/vc++, Mac. OSX/gcc, 05/10/05 LCG Applications Area Plans 5

AA Organization er Und 05/10/05 n io evis R LCG Applications Area Plans 6

AA Projects u SPI – Software process infrastructure (A. Aimar -> A. Pfeiffer) – Software and development services: external libraries, savannah, software distribution, support for build, test, QA, etc. u ROOT – Core Libraries and Services (R. Brun) – Foundation class libraries, math libraries, framework services, dictionaries, scripting, GUI, graphics, etc. u POOL – Persistency Framework (D. Duellmann) – Storage manager, file catalogs, event collections, relational access layer, conditions database, etc. u SIMU - Simulation project (G. Cosmo) – Simulation framework, physics validation studies, MC event generators, participation in Geant 4, Fluka. 05/10/05 LCG Applications Area Plans 7

SEAL and ROOT Merge u u u The main change described in the AA plan document is the merge of the SEAL and ROOT projects. The change was proposed, reviewed in depth and subsequently endorsed in the AA internal review that took place at the beginning of April. The main motivations for this merger have been the following: – to optimize use of resources by avoiding duplicate development – to provide a coherent set of products developed and maintained by the Application Area for the benefit of its clients i. e. the LHC experiments – to have the ROOT activity at CERN fully integrated in the LCG organization (planning, milestones, reviews, resources, etc. ), – to ease the long-term maintenance and evolution of a single set of software products. u Details of the merge have been discussed in the AF 05/10/05 LCG Applications Area Plans 8

SEAL and ROOT Merge (2) u The merge consists of two parts: 1. merge of the development teams into a single team 2. evolution of the software products into a single set of core software libraries u u (1) happened immediately (Rene Brun as project leader) (2) was originally foreseen to be completed within a very aggressive timescale but this has now been adapted to allow more time for the experiments to migrate their software – Experiments [ATLAS, CMS and LHCb] requested to continue the maintenance of the SEAL libraries – SEAL work package added to the ROOT project 05/10/05 LCG Applications Area Plans 9

Smooth transition for Experiments time/versions SEAL Expts S/W ROOT functionality u u Adiabatic changes towards the experiments SEAL functionality will be maintained as long as the experiments require 05/10/05 LCG Applications Area Plans 10

Merging functionality (sorted by priority) Functionality Status Math libraries Math. Core and Math. More in ROOT 5 Vector library New and included in Math. Core library Linear Algebra Disagreements between ROOT team and experiments. Possible solutions being studied Plugin manager Disagreements. Delayed migration. Using SEAL for the time being. Dictionaries Ongoing adaptation of ROOT to Reflex and Cintex in ROOT 5 Python Bindings Converged to the Py. ROOT solution. Ongoing evolution. Component Model Using SEAL component model for the time being AIDA support No work done. Not in the plans 05/10/05 LCG Applications Area Plans 11

Core libraries and services (ROOT) u u Provides basic functionality needed by any application Current work packages – BASE: Foundation and system classes, documentation and releases – DICT: Reflexion system, meta classes, CINT and Python interpreters – I/O: Basic I/O, Trees, queries – PROOF: parallel ROOT facility, xrootd – MATH: Mathematical libraries, histogramming, fitting – GUI: Graphical Uner interfaces and Object editors – GRAPHICS: 2 -D and 3 -D graphics – GEOM: Geometry system – SEAL: Maintenance of the existing SEAL packages 05/10/05 LCG Applications Area Plans 12

ROOT Milestones (level 2) BAS 1 Base 01/08/2005 New updated plugin manager BAS 2 Base 15/07/2005 A new edition of the ROOT Users Guide BAS 3 Base 15/07/2005 The ROOT bug tracking system moved to Savannah DIC 1 Dictionary 30/06/2005 Reflex and Cintex in the ROOT v 5 development release DIC 4 Dictionary 30/04/2006 CINT fully converted to Reflex DIC 5 Dictionary 31/03/2006 Py. Root exclusively based on Reflex PRF 7 Proof 30/09/2005 Demonstration of new PROOF system at ROOT 2005 MAT 1 Math 30/06/2005 First version of new Math. Core library MAT 2 Math 31/08/2005 First version of the Math. More and random number generator library MAT 3 Math 30/09/2005 New C++ Minuit in be released in ROOT MAT 4 Math 31/12/2005 Finalization of the Fitting and Minimization interfaces MAT 5 Math 31/12/2005 Root. Fit fully integrated (in CVS by July) MAT 6 Math 31/12/2005 Evaluation of a prototype linear algebra package independent of TObject GRA 1 Graph 30/09/2005 Demonstrate the new GL viewer at the ROOT workshop. SEAL 1 Seal 30/11/2005 Remove from new SEAL releases Reflex and Cintex packages. Bug fix releases will continue to include them. SEAL 2 Seal 30/11/2005 Remove from new SEAL releases Minuit and Math. Core. Bug fix releases will continue to include them SEAL 3 Seal 31/12/2005 Remove from new SEAL releases Reflection, Reflection. Builder and the Scripting packages (Py. LCGDict and Py. LCGDict 2) LCG Applications Area Plans 05/10/05 13

Software Process Infrastructure (SPI) u The AA projects share a single development infrastructure provided by the SPI project – Crucial for fostering homogeneity and avoiding duplications – Users are the LCG AA projects, LHC experiments and others external projects – Provides a number of “services” u Services – – External Software service Software distribution service Savannah portal Documentation and web site 05/10/05 LCG Applications Area Plans 14

SPI Milestones (level 2) SPI 01 31/10/2005 Provide the tools for generating CMT and SCRAM configurations from a common generic configuration description based on XML description files. Be able to update the web and distribution's kits from the same description. SPI 02 30/11/2005 Independent build and release of partitions of the LCG software SPI 03 31/07/2005 Pacman caches for binaries and sources SPI 04 31/10/2005 Automated distribution system in place, generate from XML description External Software. SPI 07 30/11/2005 Import/export features available in the Savannah system. Provide mechanism to import/migrate external data into savannah. SPI 08 31/12/2005 Savannah user forum setup, user and admin documentation available. SPI 09 31/10/2005 QA reporting on savannah and test coverage available and documented. SPI 10 30/09/2005 Testing tools updated to the latest versions on the AFS area and available to LCG projects and outside. SPI 11 01/08/2005 Doxygen and LXR automated and documented for the LCG software and for the packages (ROOT, AIDA, CLHEP) SPI 12 28/02/2006 Provide a web based "user discussion forum" service interfaced with Savannah. This new service should allow projects and experiments to easily setup and manage discussion subjects. 05/10/05 LCG Applications Area Plans 15

Persistency Frameworks (POOL) u Domain of expertise in data persistency, data management, deployment in the Grid and (relational) databases in general u The POOL project delivers a number of “products” – POOL, RAL, COOL u Work – – – packages Object storage and references Collections and Metadata Database access and distribution Catalog and Grid integration Conditions Database 05/10/05 LCG Applications Area Plans 16

POOL Milestones (level 2) POOL 01 Storage 31/12/2005 Reflex production release (Reflex validation by experiments completed) POOL 02 Storage 31/03/2006 Overall performance study and validation with expected requirements RAL 03 Ral 31/08/2005 RAL validation release (implementing the RAL review outcome and separate packaging) RAL 05 Ral 31/10/2005 RAL production release RAL 06 Ral 30/11/2005 Production release of proxy-cache access plugin CAT 02 Catalog 31/08/2005 Production release of LCG catalog adaptors (level 2) CDB 01 Cond. DB 30/04/2005 First COOL production release CDB 08 Cond. DB 15/10/2005 COOL 1. 4. 0 release with bulk insertion and retrieval in different "channels" within the same "folder" CDB 09 Cond. DB 15/10/2005 COOL passes all tier 0 validation tests CDB 12 Cond. DB 31/12/2005 First prototypes of API and command line tools for data extraction and cross-population between COOL databases 05/10/05 LCG Applications Area Plans 17

Simulation Project u u Wide range of activities in the domain of expertise of event generators and detector simulation Coordination effort to validate the physics of the simulations Experiment Validation Simulation Project Leader Geant 4 Project Fluka Project Framework WP WP WP 05/10/05 Geant 4 WP WP WP Fluka WP WP Garfield WP WP LCG Applications Area Plans Physics Validation WP WP WP MC 4 LHC Generator. S ervices WP WP 18

Simulation Milestones (level 2) SF 530 Framework 30/09/2005 Flugg application to one of the LHC calorimeter test-beam simulation SF 537 Framework 15/12/2005 Introduction of modularization of GDML files SF 538 Framework 15/12/2005 Proposal for universal handling of Monte Carlo truth SF 539 Framework 15/12/2005 Evaluation of Virtual. MC for usage in Geant 4 -Fluka test-beam validation studies SF 608 Framework 30/06/2006 Refinement to GDMLSchema to support user extensions of elements SF 609 Framework 30/06/2006 Completed evaluation of VMC for use in physics validation SF 613 Framework 31/10/2006 First release of a common framework for MCTruth handling SF 615 Framework 15/12/2006 CPPGDML code revision and re-engineering to allow easier extensibility VD 522 Validation 30/09/2005 3 rd simple benchmark for physics validation completed VD 524 Validation 30/09/2005 Validation of Fluka against one LHC calorimeter test-beam VD 532 Validation 31/10/2005 First results of ATLAS combined and 2004 test-beams data comparisons VD 612 Validation 31/10/2006 Validation of shower parameterization packages completed G 4523 Geant 4 30/09/2005 Development release G 4531 Geant 4 31/10/2005 Improved regression suite for release validation and testing infrastructure. G 4542 Geant 4 20/12/2005 New public scheduled release including positron annihilation and geometry voxelisation improvements. 05/10/05 LCG Applications Area Plans 19

Simulation Milestones (level 2) G 4605 Geant 4 31/03/2006 Development release including new tool for overlap detection at geometry construction and extensions to QGS. G 4606 Geant 4 31/05/2006 Development release. Including new features for parallel navigation enabling scoring charged particles at arbitrary locations G 4611 Geant 4 30/09/2006 Geant 4 development release. Including developments including redesign of Binary Cascade's field transitions, additional benchmarking for radiation … G 4614 Geant 4 01/12/2006 Geant 4 development release. Including surface tolerances tuned to model geometry size G 4701 Geant 4 01/06/2007 Geant 4 development release. Including refined models for EM interactions of exotic particles, first implementation of tessellated BREP solids GF 540 Garfiled 15/12/2005 Interface with a new version of Magboltz GF 541 Garfiled 15/12/2005 Interface with a new version of Heed GS 535 Genser 15/12/2005 Generator level production framework: production quality release GS 602 Genser 31/03/2006 Introduction of MCDB Grid certificates and Management of large files GS 616 Genser 15/12/2006 Fully operational LCG Generator production centre integrated in the gridmiddleware GS 701 Genser 30/06/2007 Generator level validation framework production version GS 702 Genser 30/09/2007 Pythia 8. Release of production version 05/10/05 LCG Applications Area Plans 20

Resources 05/10/05 LCG Applications Area Plans 21

Conclusions u Planning is continuous activity. Publishing the plan document is not sufficient – Tasks, resources, milestones need to be revised often – Tracking progress must be done internally and externally u Plan revisions/updates – Quarterly updates as part of the LCG quarterly report – Annual updates per project also foreseen – Annual/18 months updates in “AA internal reviews” u u Detailed planning (e. g. scheduling bug fix releases, configuration changes, etc. ) discussed and agreed at Architects Forum meetings I would like to thank all the project leaders and all the people that contributed in the preparation of the AA plan. 05/10/05 LCG Applications Area Plans 22