Testing and Release Procedures Tools Cristina Aiftimiei INFN-CNAF Mario

Testing and Release Procedures/Tools Cristina Aiftimiei (INFN-CNAF) Mario David (LIP)

SQA • SQA definition – Main effort so far; no big changes expected from now on • Base definition => D 3. 1 • Last updates and enhancements => WP 3 wiki – Procedures on 1) code contribution, 2) using external infrastructures, 3) issue tracking • SQA monitoring – Where work is focused now – Report generation, dashboard deployment (as part of T 3. 3)

SQA definition: external infrastructures

SQA definition: code contributions Procedure in WP 3 wiki: https: //goo. gl/i. Os 4 g. Q • PT using INDIGO-Data. Cloud infrastructure – Workflow: designed around SQA fulfillment » » master protected working with pull requests • PT using an external infrastructure – Workflow: code must be available in INDIGO-Data. Cloud Git. Hub organization » No specific procedure defined: forks, hooks, . .

SQA: issue tracking Git. Hub will host all the INDIGO-Data. Cloud related development issues • Easier to follow them up – EMT meetings, etc. • Easier to categorize them – Git. Hub labels • Easier to gather stats/metrics – Metrics. Grimoire not support the whole set of external tools used by PTs

Monitor SQA: metrics work in progress • Through metrics (T 3. 3 tools) – gathered from sources: • Git. Hub repositories and issue tracker – Metrics. Grimoire: fully deployed at IFCA, to be migrated to CERN • Jenkins CI reports – stored in a Metrics DB • figure out whether all of them can coexist in the same database – displayed in a dashboard • Viz. Grimoire: partially deployed at IFCA (not all the products are there), to be migrated to CERN

Metrics Dashboard ooi (Openstack OCCI Interface) dashboard Tool: Viz. Grimoire • Repository and issue tracking metrics. – Source: Metrics. Grimoire – extended to display metrics from other sources. • One page per PT will be available.

Monitor SQA: reports work in progress • Producing SQA reports – per INDIGO-Data. Cloud product – containing at least metrics defined in WP 3. 1 deliverable

Software release and maintenance INDIGO-Data. Cloud RIA-653549 9

Activities Software Maintenance & Support • Coordination of corrective, adaptive, perfective and preventive maintenance activities • Definition of • Software Maintenance process - software preparation and transition activities. • Problem Management process - analyze and document problems • Change Management process - control code, configuration changes, retirement calendars • The support to released software. • WP 3. 2 WIKI Release Management • define release plans and enforce the agreed software release policies to the software services and components delivered by WP 4, WP 5 and WP 6 • Publish release schedules • manage project public repositories of artefacts - packages and/or images • Continuous Integration -> integrating, building, and testing -> Integration Infrastructure (T 3. 3) • Continuous Delivery -> deploying on the Testing Infrastructure (T 3. 3), performing both automated and manual user acceptance tests and getting feedback from the user communities (WP 2). 10

Maintenance • Ensure the efficient and transparent management of the maintenance process providing software problems analysis and resolutions • Includes - INDIGO Releases definition, versioning • Desc. of release task item in the Release Tracking System • ID, component name, due date, list of Rf. C (features, bugs), list of packages x OS. , documentation, license, release notes • release task state transition DONE INDIGO-Data. Cloud RIA-653549 11

Change management (2) 18/03/2018 • Rf. C • Features and/or improvements • Development tasks • WP 4 - http: //bit. ly/indigo-wp 4_tasks =119 • WP 5 - http: //bit. ly/indigo-wp 5_tasks = 35 • WP 6 - http: //bit. ly/indigo-wp 6_tasks = 66 • Bugs • INDIGO-DC Git. Hub Dashboard INDIGO-Data. Cloud RIA-653549 12

Support § Organises § Support for users – GGUS § Escalation procedures § Ensures § timely solution of tickets § communication with developers (WP 4, 5, 6) § Provides § statistics on ticket processing in relation to, eventual, SLA § Activity § Set-up GGUS SU – “INDIGO - Data. Cloud Catch-All” § 2015 -09 -30, Level of Support = Base (5 working days) § Set-up indigo-support mailing-list - indigo-su@lists. indigo-datacloud. eu § TODO - Define monitoring procedure according to the Technology_Provider_Corporatelevel_UA_v 1. 0_Final. docx INDIGO-Data. Cloud RIA-653549 13

EMT & PTs § What is the exact interface between developers (WP 4, 5, 6) and the Release Manager? Ø EMT § In order to manage the release process, maintenance and support § led by the release manager and composed by nominated experts of WP 2, WP 4, WP 5, WP 6, quality assurance and pilot services. § Activities § § § EMT To. R EMT mailing-list - indigo-emt@lists. indigo-datacloud. eu EMT agendas & meetings PTs & components Github & Docker. Hub – organization, teams, members INDIGO-Data. Cloud RIA-653549 14

Release Management § 2 Major Project releases (month 16 & 24) § components, available on a predefined set of platforms § Volume available for repositories of packages § Docker. Hub for containers INDIGO-Data. Cloud RIA-653549 15

Services for SQA and CI Public services • Version Control System https: //github. com/indigo-dc – 28 members registered – grouping into teams (10 at the moment) – 16 repos • Github dashboard and issue tracker – https: //github. com/orgs/indigodc/dashboard • Docker repository https: //hub. docker. com/u/indigodatacloud/ – 10 members – 1 repository 16

Services for SQA and CI WP 3 hosted services • jenkins-CI deployed at CERN • The other elements following 17

Services continuity plan for migration and recovery The WP 3. 3 services for SQA and CI are deployed in the CERN cloud (Note that the services are now being implemented/deployed): • Continuous Integration service: Jenkins-CI – jenkins. indigo-datacloud. eu • Artifacts Repository – repo. indigo-datacloud. eu • Code metrics services: Metrics. Grimoire and Viz. Grimoire – metricg. indigo-datacloud. eu – vizg. indigo-datacloud. eu More details are being discussed in the working document: https: //docs. google. com/document/d/16 unk 7 b. Yrx 1 EVWwfta. X 458 p 6 H 70 NRL 7 JOs. IQYp 8 NCqk/edit# 18

Services continuity plan for migration and recovery These services need a backup or replica at another partner site: • Partner sites – CERN is the master/active site where the services will be primarily exposed to Indigo. – CNAF is the slave/passive site hosting a replica/backup of the services • Virtual Machines (light blue boxes in diagram) – There are 4 VM images with the services: jenkins, metricsg, vizg, repo – Snapshots will be produced and transferred between sites • Storage/Data volumes (light red boxes in diagram) – /var/lib/jenkins: rsync of this directory between the master/active site and the slave/passive site. – My. SQL (or Maria. DB): A master/slave architecture. – The YUM and APT repositories: mirroring the repositories. More details are being discussed in the working document: • Network and certificates https: //docs. google. com/document/d/16 unk 7 b. Yrx 1 EVWwfta. X 458 p 6 H 70 NRL 7 JOs. IQYp 8 NCqk/edit# – DNS alias – Certificate will contain DNS alias and real DNS 19 dependent from the site

Testing infrastructure • WIKI: https: //project. indigodatacloud. eu/projects/wp 3/wiki/Testing_infrastructure – Procedure to request resources for SW development from PTs – List of resources for Indigo components • Gathers a list of resources that PTs use for their SW development – VO indigo: has been deployed by LIP, contains the information about the VO and VOMS server 20