EGEE First Conference Cork April 19 2004 www

EGEE First Conference, Cork, April 19, 2004 www. eu-egee. org JRA 1 Testing Activity: Status and Plans Leanne Guy EGEE Middleware Testing Manager EGEE is a project funded by the European Union under contract IST-2003 -508833

Contents • Middleware testing activity § Mandate, milestones and deliverables • Test team structure • Scope of the middleware testing activity • Current tasks, activities and progress to date § Testbed setup, Tools assessment, Test plan preparation • Relationship with other activities § NA 4, SA 1, JRA 3, ARDA, Globus EGEE First Conference, Cork, April 19, 2004 - 2

Testing activity mandate • Test all middleware components that form part of the integrated middleware system to ensure a production quality release that fulfills the requirements of the applications. • Assess that all software requirements have been correctly and completely implemented and are traceable to system requirements, • Test the integrated software for scalability, platform independence and stress resilience EGEE First Conference, Cork, April 19, 2004 - 3

Milestones and deliverables – Year 1 Month Deliverables & Milestones M 03 June’ 04 MJRA 1. 1 Testing tools are available to testing, integration and implementation teams. M 03 June’ 04 MJRA 1. 2 Distributed testing infrastructure available – hardware allocated and software being installed M 05 August’ 04 MJRA 1. 3 Integration and testing infrastructure in place and operational. Continuous integration and testing begins. Test plan for release candidate 1 core middleware components available. M 09 December’ 04 MJRA 1. 4 First integrated release candidate 1 available. Integration test suite must be available in time for the delivery of middleware components. M 10 January’ 05 MJRA 1. 5 Integrated release 1 candidate enters final testing and validation period. M 12 March’ 05 DJRA 1. 3 First tested major release of EGEE middleware delivered to SA 1 including test suites and reports Item EGEE First Conference, Cork, April 19, 2004 - 4

Test team structure (I) • Testing activity must be distributed across at least 3 sites § Minimum to test all basic grid functionality § Ensure that assumptions valid at only one site have not been made • Requirements on sites § 1 sysadmin dedicated at least 50% to the activity § Deploy candidate releases (~weekly) § Diagnose and report installation problems – provide fixes § Sysadmin task at CERN shared with Integration team § At least 15 machines dedicated to the testing and validation activity § Support for the primary platform – RHES 3. 0 § If support for secondary platform not possible we will need to look to other sites to participate EGEE First Conference, Cork, April 19, 2004 - 6

Test team structure (II) The 3 initial testing sites are CERN, NIKHEF and RAL § More sites can join the testing activity at a later stage ! § Must fulfil site requirements • CERN: § Main testing site, responsible for coordination and management § Maite Barroso Lopez: Integration and testing manager § Leanne Guy: Test team manager § Diana Bosio, David Collados, Mario Reale : Testers • NIKHEF § Kors Bos, David Groep, Davide Salomi : sys admin • RAL § Steve Traylen: sys admin EGEE First Conference, Cork, April 19, 2004 - 7

Scope of the testing activity (I) • Unit testing § Implementation clusters responsible for providing unit tests § Code coverage analysis tool to assess extent of unit testing § Integrate into the build system § Components failing unit tests or not sufficiently tested will not be integrated § Metrics to be defined § Unit tests will be validated on testing testbed § Include human code reviews • API testing § Ensure that all APIs conform to the service specifications § Defined by whatever the architecture team (or equivalent) is ! § Automate and integrate into the build system § Reject components that deviate from the reference definition EGEE First Conference, Cork, April 19, 2004 - 8

Scope of the testing activity (II) • Functional testing § Fully integrated system validated against the functional requirements as specified in user requirements documents. § Verify that all the required functionality has been correctly and completely implemented § Data management, Job submission, Information publication, etc § Error recovery and reporting § Driven by application uses cases § Each application use case will be directly associated with a testsuite § Overlap with NA 4 application testing team § Core of the test team’s work ! EGEE First Conference, Cork, April 19, 2004 - 9

Scope of the testing activity (III) • Regression testing § Essential to ensure that bugs fixed in one release do not reappear in a later release § Automate as much as is possible and include in the build • Security testing § Secure access to data especially biomedical requirements! § Will need help from JRA 3 to design security tests ! • Platform Independence testing § Need to demonstrate the portability of the middleware § Fully test and validate middleware on the main platform § Demonstrate portability only on the secondary platform. EGEE First Conference, Cork, April 19, 2004 - 10

Scope of the testing activity (IV) • Documentation testing § Initial input provided to integration team by implementation clusters § Integration team will consolidate user documentation from developers § Release and installation documentation will be produced by integration team § Test team will verify all official documentation • Installation and configuration testing § Based on the installation guides provided by integration team § Tool independent installation and configuration of the full system will be tested by the test team. EGEE First Conference, Cork, April 19, 2004 - 11

Scope of the testing activity (V) • Scalability, Stress resilience, Performance testing § Will begin later in the year on a candidate release that is deemed to be sufficiently stable, § Testing on a given release in parallel with the continuous integration and testing process § Overlap with NA 4 testing ? • External software testing ? § When we find bugs do we simply report them to the vendor or do more tests § We do not have the resources to extensively test external components We will make use of existing testsuites from EDG and LCG EGEE First Conference, Cork, April 19, 2004 - 12

Current status of the testbed • 5 Machines at CERN, 2 at NIKHEF, 2 at RAL available • Core middleware components of the first middleware prototype for ARDA being installed on the testbed at the moment: § Castor SRM, GFAL, Alien, R-GMA, LRC, VOMS, My. Proxy, LCMAPS/LCAS • Quattor will be used by all sites for testbed management § § SA 1 certification and validation team will also use Quattor NIKHEF and RAL also have Quattor installations already Quattor installation at CERN for JRA 1 underway Quattor objects for middleware prototype components to be written MJRA 1. 2: Distributed testing infrastructure available – hardware allocated and software being installed EGEE First Conference, Cork, April 19, 2004 - 13

Middleware test plan • Testing activities will be driven by the testplan document • Test plan being developed based on user requirements documents: § Application requirements from NA 4: HEPCAL I&II, AWG documents, Bio-informatics requirements documents from EDG § Deployment requirements being discussed with SA 1 § ARDA working document for core Grid services § Security: work with JRA 3 to design and plan security testing • The test plan is a living document: it will evolve to remain consistent with the evolution of the software MJRA 1. 3: Test plan for release 1 core components made publicly available beginning September 2004. EGEE First Conference, Cork, April 19, 2004 - 14

Other current activities • Assessment of tools to be used for testing middleware components (Diana Bosio) § Intend to leverage the experience of LCG and EDG, § Support for tools from SPI § Web page of the evaluation at http: //cern. ch/egee-jra 1 - testing/tools. html • Preparation of automated test procedures integrated with the build system (not started) § Automate as much as is possible all middleware testing § Establish a continuous integration and testing process • Preparation of test data (not started) • Plan for hardware resources needed • Set up CVS repository and savannah for task and bug tracking EGEE First Conference, Cork, April 19, 2004 - 15

Relationship with other activities • ARDA § First prototype instance (end April’ 04) will be used to set up the testing infrastructure only, it will not be tested prior to release § Timelines too tight § Delays in the starting date of ½ the test team to 1 May § Implementation and testing activities need to converge soon! § HEP specific testing activities need to be coordinated with ARDA § Currently the first prototype is being installed on the testbed § Test and evaluate tools and gain experience § Develop Quattor objects § Maybe provide feedback but no commitment initially EGEE First Conference, Cork, April 19, 2004 - 16

Relationship with other activities • SA 1 Certification and Testing § Overlap in deployment and certification testing § Will share testsuites for validation of basic functionality § Collaborate on Quattor tasks • NA 4 Test Group § Common goals around functional, stress, performance testing. § Meeting arranged at Cork to discuss plans to collaborate • Globus § Meeting with Lisa Childers - common desire to share ideas on testing grid services § Work on a common definition of terminology for testing grid services EGEE First Conference, Cork, April 19, 2004 - 17