Software Integration and Test Techniques in a Large

Software Integration and Test Techniques in a Large Distributed Project: Evolution, Process Improvement, Results Paola Sivera - ESO ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN

Chajnantor Plateau – looking north V. Licancabur Cº Chajnantor Pampa La Bola Center of Array ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 2

Antenna Prototypes at ALMA Test Facility: 2005 ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 3

ALMA Geographical distribution Univ. Calgary ALMA ATF c DRAO Penticton MPI Bonn ATC Edinburgh Jodrell Bank ESO c DAMIR/IEM Madrid NAOJ NRAO ALMA Santiago Obs de Paris IRAM Grenoble Brera Observatory Arcetri Observatory ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 4

Trilateral Computing IPT Organisation Total Bilateral staff now: 40 FTEs ALMA Management B. Glendenning G. Raffi K. Tatematsu Science Software Requirements R. Lucas Executive P. Grosbol Control A. Farris High Level Analysis J. Schwarz Software Eng M. Zamparelli Correlator J. Pisano Scheduler A. Farris Integration P. Sivera Total Bilateral staff now: 65 FTEs trilateral staff now: 40 FTEs Total trilateral staff now: 65 FTEs Common SW G. Chiozzi Observation Preparation A. Bridger Pipeline L. Davis Archiving A. Wicenec Offline J. Mc. Mullen ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN ACA M. Watanabe Operations Support M. Chavan Telescope Calibration R. Lucas 5

ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 6

ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 7

ALMA System Dataflow ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 8

Obs. Prep Visual Editor (to select spectral setup) ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 9

EXEC: Operator Master Client ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 10

CASA Viewer • Visibility displays (raw, corrected data) • Image, data cube displays – Statistics; profiles; raster, contour, vector support ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 11

Standards and Tools • Alma Common Software • Environment • CVS organization • Makefile • Standard Test Environment (STE) • Tool for automated testing • Bug tracking tool (JIRA) ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 12

ALMA Release Cycle ICD Freeze = 15 th of the month Monthly integration = last day of the month Minor Release: Mar 31 • • • Major Release: Sep 30 • • • Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 13

Lessons learned • Releases twice per year but… • Subsystem developers working in isolation • Results thrown over the wall to ITS at release time – Each integration took ~2 months – At completion of integration code was already obsolete ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 14

Function-based Teams ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 15

FBTs organization Z Shared. Simulator-2006 -02 -B (branch) Shared. Simulator-2006 -02 -Root. Of. Branch (tag of the HEAD) A Shared. Simulator-2006 -02 -Before. Merge B TRUNK-Shared. Simulator-2006 -02 -Before. Merge C TRUNK-Shared. Simulator-2006 -02 -After. Merge (tag of the HEAD) Shared. Simulator-2006 -02 -After. Merge-B (branch) F Shared. Simulator-2006 -02 -FINAL ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 16

Summing it up • FBTs advantages: – Improve communication – More effective involvement of testers • Testing end-to-end since the beginning – Good for object-oriented software – A lot of maintenance on the test cases – It is not enough -> user tests ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 17

Where we are Testing maturity (according to Boris Beizer) 1. 2. 3. 4. there is no difference between testing and debugging: often during FBTs’ work the purpose of testing is to show that the software works: at FBTs validation the purpose of testing is to show that the software does not work: release validation the purpose of testing is not to prove anything, but to reduce the perceived risk of not working to an acceptable value: release validation ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 18

Future developments • Make software more testable from its inception • adoption of the test maturity model (TMM) • … the list can be very long … For the near future we want to focus on: • Stability tests • More automation ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 19

AOS Technical Building ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 20

ALMA - 2012 ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 21

ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 22

MELCO : August 2007 ICALEPCS 2007, 15 -19 Oct 2007, Knoxville TN 23