Qualitätssicherung von Software SWQS Prof Dr Holger Schlingloff

Qualitätssicherung von Software (SWQS) Prof. Dr. Holger Schlingloff Humboldt-Universität zu Berlin und Fraunhofer FOKUS 15. 7. 2014: Modellbasierter Test (Jaroslav Svacina)

Specification-based Testing • Constructing the test suite from the specification § as opposed to constructing it from the implementation (= code-based testing) specified behaviour implemented behaviour tested behaviour § code-based testing cannot detect missing requirements § specification-based testing cannot detect additional (unspecified) features H. Schlingloff, Software-Qualitätssicherung Folie 3

Model-based Testing • • • SUT is compared to a formal model (requirements) Automation of test case design Early validation of requirements Traceability Easy maintenance of test suites for regression testing H. Schlingloff, Software-Qualitätssicherung Folie 4

Model-based Design and Model-based Testing Requirements System Spec Test Spec System Model Test Model System Impl. Test Cases • Often: same syntax, different pragmatics § e. g. test cases can be formulated in Java § e. g. system spec can be formulated with LTL H. Schlingloff, Software-Qualitätssicherung Folie 5

System models vs. Test models • Models can help in the development of complex • systems The more concrete the formalism, the closer it is to an implementation § executable code may be generated from state diagrams § We might additional information such as timing, communication, variables and such. • Test model as opposed to implementation model describes properties of the targeted system § not aiming at a complete description of the system § not aiming at the generation of executable code H. Schlingloff, Software-Qualitätssicherung Folie 6

Model-based Testing H. Schlingloff, Software-Qualitätssicherung Folie 7

Model-based Testing H. Schlingloff, Software-Qualitätssicherung Folie 8

Modelling • Describe system requirements for test generation • High abstraction level H. Schlingloff, Software-Qualitätssicherung Folie 9

Modelling Notations • Pre/Post (state based) notations § State variables, operations (pre-/postcondition) § B, Spec. Explorer (c#), UML OCL • Transition based notations § Focus on describing the transitions between states of the system § UML State Machines, Simulink State Flow • … H. Schlingloff, Software-Qualitätssicherung Folie 11

Example: A Kitchen Toaster • A toaster § what is the technical process? § what are the states, events and signals of the (technical) process? § what are the boundaries of the system? § which information processing is to be done? § what are the interfaces between technical system and information processing component? H. Schlingloff, Software-Qualitätssicherung Folie 12

Modeling the Toaster • User Interfaces: turning knob, • • side lever, stop button Internals: heating element, retainer latch Extra: defrost button • First approach: timing is neglected (timer event) • Advanced approach: timing depends on various parameters H. Schlingloff, Software-Qualitätssicherung Folie 13

Toaster – Simple State Machine H. Schlingloff, Software-Qualitätssicherung Folie 14

Toaster – Hierarchical Design H. Schlingloff, Software-Qualitätssicherung Folie 15

Toaster – with Variables H. Schlingloff, Software-Qualitätssicherung Folie 16

Test Generation H. Schlingloff, Software-Qualitätssicherung Folie 17

Test Generation • Algorithm § Graph traversal yields abstract test cases - Dijkstra shortest path algorithm - Depth-first and breadth-first search § Evolutionary algorithm • Model Transformation • Static analysis for input parameters § Partition Testing § Classification tree method § Boundary value analysis H. Schlingloff, Software-Qualitätssicherung Folie 18

All-States with Dijkstra and DFS H. Schlingloff, Software-Qualitätssicherung Folie 19

Evolutionary Test Generation H. Schlingloff, Software-Qualitätssicherung Folie 20

Test Selection H. Schlingloff, Software-Qualitätssicherung Folie 22

Test Generation from State Machines • Define a test case to be any execution path • How to generate such paths? How many paths to generate? When to stop testing? Coverage criteria e. g. § § all-states all-transition-pairs all-n-paths • Test goal: one particular item to be covered H. Schlingloff, Software-Qualitätssicherung Folie 23

Tests for State Machines • For a state machine, a test case is just a finite • • sequence of external triggers and actions A test goal is a particular entity of the state machine (region, pseudostate, transition, n-path, …; for each test and goal it is defined whether the test reaches this goal The coverage of a test suite is the percentage of reached test goals § the coverage can either be measured during the execution of a test suite, or statically before execution H. Schlingloff, Software-Qualitätssicherung Folie 25

Coverage for State Machines • common coverage criteria for UML state machines all-states all-transitions all configurations: all combinations of parallel substates n-transition-coverage means all reachable transition sequences of length n are covered (esp. : pairs) § All-loop-free-paths § All-n-loop-paths § decision coverage, condition coverage, MC/DC, . . . § § • all-requirements H. Schlingloff, Software-Qualitätssicherung Folie 26

Szenarien als Testselektionskriterium • Schwierige, nicht ausreichende Möglichkeiten zur Beschreibung der Testselektionskriterien • Einflussnahme der Domänenexperten auf den Testselektionsprozess H. Schlingloff, Software-Qualitätssicherung Folie 27

Szenarien als Testselektionskriterium H. Schlingloff, Software-Qualitätssicherung Folie 28

Automated Test Generation Tools • More than a dozen commercial and • • • experimental research tools available Usually quite costly (>10 K€ per license) Mostly from UML State Machines Comparison e. g. by mutation analysis H. Schlingloff, Software-Qualitätssicherung Folie 29

Conformiq H. Schlingloff, Software-Qualitätssicherung Folie 30

Par. Te. G – Partition Test Generator • Since we didn’t like the pricing, and wanted to experiment with different technologies, a Ph. D. student built his own… § http: //parteg. sourceforge. net/ • UML class & state transition diagrams, connected by OCL • Plugin for Eclipse, supports XMI import / export H. Schlingloff, Software-Qualitätssicherung Folie 31