INFSO-RI-223782 Quality Assurance with ETICS multinode automated

INFSO-RI-223782 Quality Assurance with ETICS – multinode automated testing CGW 09 M. Zurek, A. A. Rodriguez, A. Aimar, A. di Meglio, L. Dini CERN Krakow, 12 October 2009

Contents INFSO-RI-223782 • ETICS Overview • Multi-Node Testing • Q&A ETICS Testing Tools CGW 09 2

INFSO-RI-223782 ETICS Overview ETICS Testing Tools CGW 09 3

The Problem INFSO-RI-223782 Distributed Computing, Distributed Development ETICS Online tutorials - Module 01 4

The ETICS System INFSO-RI-223782 • Continuous software build, test and QA verification system • Easily extensible with additional plugins • Scheduled or on-demand build and test jobs • User computers or remote distributed infrastructures • Built-in connectors to distribute build and test jobs on different types of infrastructures from standard job management systems to the grid • Open repository of configuration metadata, packages and build, test and QA reports • Support for standard package management systems like YUM • It’s multi-platform and independent from any specific language, build or test tool ETICS Testing Tools CGW 09 5

Architecture ETICS Testing Tools INFSO-RI-223782 CGW 09 6

The Configuration Web Client INFSO-RI-223782 ETICS Online tutorials - Module 01 7

Command-line Client INFSO-RI-223782 ETICS Online tutorials - Module 01 8

The Repository Web Application INFSO-RI-223782 ETICS Online tutorials - Module 01 9

INFSO-RI-223782 Examples of Plugins Metrics Type Programming languages/ technologies Complexity static Java Python Javancss JCcn. Plugin Py. Complexity. Plugin. py Design quality static Java Jdepend JDepend. Plugin N of potential bugs static C/C++ Python Perl PHP Java Flawfinder, RATS PMD Findbugs CFlawfinder. Plugin CPy. Php. Rats. Plugin JPmd. Plugin JFindbugs. Plugin N of potential bugs dynamic C/C++ Valgrind CValgrind. Plugin static All SLOCCount. Plugin Coverage dynamic Java Emma Cobertura JUnitemma. Plugin JCobertura. Plugin Unit tests success rate dynamic Java Python JUnit Py. Unit JUnit. Plugin JUnitreports. Plugin. py Py. Unit. Plugin. py static IPv 6 WSI Lines of code Compliance with standards Profiling dynamic ETICS Online tutorials - Module 01 C/C++ Java Tool ETICS Plugin IPv 6 Plugin WSInteroperability. Plugi n Jrat Valgrind JRat. Plugin CValgrind. Plugin 10

INFSO-RI-223782 Multi Node ETICS Testing Tools CGW 09 11

Multi-Node INFSO-RI-223782 The ability to setup scenarios where multiple services are automatically deployed in multiple nodes. These services must be able to work properly as if they would be installed manually by a user performing any required operation in the required order. ETICS Testing Tools CGW 09 12

Challange Service Synchronization ETICS Testing Tools INFSO-RI-223782 CGW 09 13

Challange Information Interchange ETICS Testing Tools INFSO-RI-223782 CGW 09 14

Challange Flow Monitoring ETICS Testing Tools INFSO-RI-223782 CGW 09 15

INFSO-RI-223782 The ETICS Solution • ETICS-SET is used to publish key-value pairs or only keys to the central information system in order to make the information available to all the other nodes, services or phases. • ETICS-GET is used to get (blocking or not-blocking) the information published by some local or remote execution of ETICS-SET. • ETICS-LOG is used to log and for real-time monitoring. Server Node Central IS [ … INSTALL …] Client Node $IP = ETICS-GET S. IP [ … CONFIG … ] [ … START … ] ETICS-SET S. IP = x. x ETICS-LOG “RUNNING” S: “RUNNING” ETICS-GET C. DONE [… RUN TEST …] C. DONE [… STOP AND EXIT …] ETICS Testing Tools [ … USE $IP … ] ETICS-SET C. DONE [… EXIT …] CGW 09 16

The PING Example (1/3) ETICS Testing Tools INFSO-RI-223782 CGW 09 17

The PING Example (2/3) ETICS Testing Tools INFSO-RI-223782 CGW 09 18

The PING Example (3/3) INFSO-RI-223782 CLIENT SERVER ETICS Testing Tools CGW 09 19

Challenges INFSO-RI-223782 • Deployment automation in the context of complex software development is a prerequisite for automatic system testing • Complexity of service configuration • Developers should provide deployable configurations, or parameterised configurations • External dependencies: all dependencies must be deployable automatically • Knowledge of these “external” components is not necessarily with the end-user • Privileged deployment/configuration operations • Root / Super User actions • Mix of normal user-level and root level actions ETICS Testing Tools CGW 09 20

Benefits INFSO-RI-223782 Automation of complex processes Automatic setup of complex environment Automatic generation of reports Save on resources and time • More scenarios can be explored • Less human errors Form the foundation for more advanced system tests ETICS Testing Tools CGW 09 21

Thanks! INFSO-RI-223782 http: //www. eticsproject. eu ETICS Testing Tools CGW 09 22