Practical Approach to Specification and Testing of Distributed

Practical Approach to Specification and Testing of Distributed Network Applications Victor Kuliamin kuliamin@ispras. ru Nickolay Pakoulin Alexander Petrenko ISP RAS, Moscow

Outline Introduction n Event Contracts Specification n Application 1 : Standards Clean-up n Application 2 : Conformance Testing n

Internals and Externals of Availability Two approaches to ensure service availability n ‘Internal’ Based on guarantees of quality of components themselves ¨ Rigorously defines obligations of participants ¨ Tries to enforces obligation fulfillment ¨ FAILED n n Systems are too complex to comprehend or formally describe as a whole Formal methods work only in completely described cases Classic formal methods need high-educated and experienced staff ‘External’ Based on external infrastructure providing availability while components can fail or go down ¨ Uses additional infrastructure and mechanisms of service delivery ¨ Provides additional means of control ¨ Imposes external restrictions on components’ operation ¨ Enforces a set of standards

Factors of Success n Standards enforcement Rigorous definitions of what is required – need for strict requirements specifications ¨ Guarantees of interoperability – consistency and unambiguity ¨ Conformance testing and certification ¨ n Constraints on specification technique used Support of iterative and component-wise development ¨ ‘Lightweight’ techniques able to get results ¨ n n ¨ Based on incomplete descriptions Without special requirements to staff Support of full-scale testing Automated test construction is preferable

How to Describe Requirements? n The description should be ¨ Sufficiently expressive ¨ As clear as possible ¨ Scalable to rather complex systems – preferably, component-wise ¨ Suitable to distributed systems – include several sides n How do people describe mutual obligations in complex cases including several parties? – By means of contracts!

Outline Introduction n Event Contracts Specification n Application 1 : Standards Clean-up n Application 2 : Conformance Testing n

Contract Specifications n Pre- and postconditions (Hoare, 1969) – means for reasoning about program behavior – augment code elements to enforce rigorous development n Design by Contract (Meyer, 1992) – software is considered as a set of components interacting trough their interfaces – pre- and postconditions are defined for interface operations – constraints on data integrity are stated in invariants – together they form software contract between a component and its environment

Design by Contract Pro and Contra n Advantages Component-wise consideration of software – support for reuse, incremental and parallel development ¨ Possibility to use for different aspects and on different abstraction levels ¨ n Drawback ¨ PU 2 Insufficient for distributed systems – does not consider concurrency and interaction PU 1 PL 1 – does not consider callbacks asynchronous

Interaction in Distributed Systems

Event Contracts Obligations of the environment Obligations of the system Precondition Pre-state System Input event says in what states such an event is possible Environment Post-state Obligations of the system Output event Environment Postcondition says what poststates can follow such an event in such a pre-state Pre-state System Post-state Obligations of the system

Concurrency ~

Implementation n Software Contracts ¨ Pre- and postconditions of events, invariants ¨ Possibility to specify constraints in form of predicates on the results, not the algorithm ¨ Component-wise consideration of software ¨ Asynchronous events and callbacks included n n Specifications in extensions of widely-used programming languages (C, Java, C#) Simplifications where possible ¨ Joint description of call and return if intermediate states do not matter

Example public specification class Barrier { int awaited. Threads = 0; int waiting. Threads = 0; invariant Counters. Are. Nonnegative() { return awaited. Threads >= 0 && waiting. Threads >= 0; } public specification void Init(int n) { post { if(n < 0 || waiting. Threads > 0) { branch No. Changes; return awaited. Threads == pre awaited. Threads && waiting. Threads == pre waiting. Threads; } else { branch New. Height. Set; return awaited. Threads == n && waiting. Threads == 0; } } } public specification void Wait() { post { if(awaited. Threads <= 1) { branch Immediate; return awaited. Threads == 0 && waiting. Threads == pre waiting. Threads; } else { deferred branch Waiting; return awaited. Threads == pre awaited. Threads - 1 && waiting. Threads == pre waiting. Threads + 1; } } deferred return { pre { return awaited. Threads == 0 && waiting. Threads > 0; } post { return waiting. Threads == pre waiting. Threads - 1; } }

The Proposed Approach Formalization Standards Contract Specifications Inconsistencies, ambiguities, interoperability flaws Testing goals Test Suites Conformance testing Certification Interoperability testing Early debugging Requirements traceability Software

Outline Introduction n Event Contracts Specification n Application 1 : Standards Clean-up n Application 2 : Conformance Testing n

Case Studies I n IPv 6 ¨ Parts considered n n ¨ Sending datagrams / receiving packets Neighbor discovery Multicast Listener Discovery UPD over IPv 6 Results n n n 2001 Minor defects found in RFC 2460 Conformance test suite developed (further) IPMP-2 ¨ 2004 Results n n n Several contradictions between standard parts found Interoperability flaws detected in Mutual Authentication protocol 2 accepted submissions on elaboration of the standard

Outline Introduction n Event Contracts Specification n Application 1 : Standards Clean-up n Application 2 : Conformance Testing n

Testing Fundamentals How to test? n We act upon the system under test n We watch its reaction n We check whether that reaction is what should be n We repeat this until all the reasonable situations are exhausted

Testing Goals post { if ( f(a, b) || g(a) ) … else if( h(a, c) & !g(b) ) … else !f(a, b) && !g(a) && !h(a, c) … || !f(a, b) && !g(a) && g(b) }

The Testing Scheme System under Test Behavior Model Testing Model Coverage Model Single Input Checking On-the-fly Test Sequence Generation

Uni. Tes. K Test Construction Tools n n n C / Visual Studio 6. 0, gcc Java / Net. Beans C++ / Net. Beans + MS Visual Studio specifications in Java extension Specialized tool for compiler testing and complex data generation C# / Visual Studio. NET 7. 1 Java / Eclipse 2002 2003 ~ 2005

Tool Demonstration

Case Studies n n ISP RAS – Nortel Networks functional test suite development for Switch Operating System kernel IPv 6 implementations ¨ ¨ ¨ n n n 2001 -2003 Microsoft Research Mobile IPv 6 (in Windows CE 4. 1) Oktet Intel compiler optimization units IPSec Pilot projects ¨ ¨ 1994 -1997 2001 -2003 2004 -… Enterprise application development framework Components of Tiny. OS Web-based banking client management system (Luxoft) Components of billing system (Vympelkom) http: //www. unitesk. com 2003 2004 2005

References 1. 2. 3. 4. 5. 6. 7. V. Kuliamin, A. Petrenko, I. Bourdonov, and A. Kossatchev. Uni. Tes. K Test Suite Architecture. Proc. of FME 2002. LNCS 2391, pp. 77 -88, Springer-Verlag, 2002. V. Kuliamin, A. Petrenko, N. Pakoulin, I. Bourdonov, and A. Kossatchev. Integration of Functional and Timed Testing of Real-time and Concurrent Systems. Proc. of PSI 2003. LNCS 2890, pp. 450 -461, Springer-Verlag, 2003. V. Kuliamin, A. Petrenko. Applying Model Based Testing in Different Contexts. Proceedings of seminar on Perspectives of Model Based Testing, Dagstuhl, Germany, September 2004. A. Kossatchev, A. Petrenko, S. Zelenova. Using Model-Based Approach for Automated Testing of Optimizing Compilers. Proc. Intl. Workshop on Program Undestanding, Gorno-Altaisk, 2003. V. Kuliamin, A. Petrenko, A. Kossatchev, and I. Burdonov. The Uni. Tes. K Approach to Designing Test Suites. Programming and Computer Software, Vol. 29, No. 6 , 2003, pp. 310 -322. (Translation from Russian) S. Zelenov, S. Zelenova, A. Kossatchev, A. Petrenko. Test Generation for Compilers and Other Formal Text Processors. Programming and Computer Software, Vol. 29, No. 2 , 2003, pp. 104 -111. (Translation from Russian) V. Kuliamin, N. Pakoulin, A. Petrenko. Extended Design-by-Contract Approach to Specification and Conformance Testing of Distributed Software. Proc. of 9 -th World Multi-Conference on Systemics, Cybernetics, and Informatics, Model Based Testing Session, July 2005, to be published.

Contacts Red. Verst group web site http: //www. ispras. ru/groups/rv/rv. html n Uni. Tes. K projects web site http: //www. unitesk. com n Group leader Alexander Petrenko petrenko@ispras. ru n

Thank You!