Скачать презентацию Nokia Research Center C A R M E Скачать презентацию Nokia Research Center C A R M E

601e888d82f4ac8410bede4eada6a7e4.ppt

  • Количество слайдов: 20

Nokia Research Center C A R/ M E M /V T T Making TTCN-3 Nokia Research Center C A R/ M E M /V T T Making TTCN-3 work! Issues and strategies for its use in product development Martin Botteck, Thomas Deiß, Colin Willcock Nokia Research Center Bochum, Germany Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 1

Nokia Research Center C A R/ M E M /V T T TTCN-3 adoption Nokia Research Center C A R/ M E M /V T T TTCN-3 adoption today: • Generally, uptake is slower than anticipated. Speed-up could be possible through : • Handling variations of the testing task • Integrating TTCN-3 into existing test environments • Using TTCN-3 in the hardware development process Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 2

Nokia Research Center C A R/ M E M /V T T Overview Variations Nokia Research Center C A R/ M E M /V T T Overview Variations in Test Systems TTCN-3 in existing test environments TTCN-3 for testing hardware Conclusion/Summary Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 3

Nokia Research Center C A R/ M E M /V T T Variations in Nokia Research Center C A R/ M E M /V T T Variations in Testing • Within a set of test cases there often are several commonalities • Example: 3 G “Call” – several types of calls (voice, data, …) and calling routes – behaviour will in many cases be similar, but not the same – “intuitive” solution: copy/paste code from one test case to another “similar” one • -> maintenance problem when the 3 G specification is updated – alternative solution: add a parameter in the test case invocation • -> code will become hard to read Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 4

Nokia Research Center C A R/ M E M /V T T Types of Nokia Research Center C A R/ M E M /V T T Types of variations • Value Variants – Variations that may be expressed by altering a parameter inside a test case or message sent to the SUT – seems trivial but soon becomes difficult due to • vast amount of parameters and combinations • typically not all combinations make sense • dependancies between the values is typically not documented and implicit – a language or coding style solution needs to be found for this Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 5

Nokia Research Center C A R/ M E M /V T T Types of Nokia Research Center C A R/ M E M /V T T Types of variations (ctd) • Behavioural Variants – Variations that may be expressed by a different sequence of messages between the SUT and the tester • several messages or sets of sequences may remain the same • some are different – decomposition of the SUT may reveal common abstract test case structures with respect to the SUT components – but what about complex composition variants of such components? – -> Abstract Modelling of the tester? (UML testing profile, Test Purpose Language) Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 6

Nokia Research Center C A R/ M E M /V T T Types of Nokia Research Center C A R/ M E M /V T T Types of variations (ctd) • Variants of the System under test – product customisation – evolution of conformance standards • Might be expressed by altering a parameter (e. g. “feature_x=OFF”, “call_type=‘rich_call’”) • in combination with reference to a set of message sequences – Typically, a vast amount of such combinations will be needed • How to solve this without copy/paste of code but still avoiding combinatorial logic to the user? Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 7

Nokia Research Center C A R/ M E M /V T T Overview Variations Nokia Research Center C A R/ M E M /V T T Overview Variations in Test Systems TTCN-3 in existing test environments TTCN-3 for testing hardware Conclusion/Summary Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 8

Nokia Research Center C A R/ M E M /V T T The dilemma Nokia Research Center C A R/ M E M /V T T The dilemma of choice • Nowadays most R&D departments already perform testing: • Conformance testing, hardware testing, product testing, module testing, unit testing… • Proprietary test systems (“home grown” or “tailor made”) • Multitude of vendors with closed environments • Trying out a system is a lot of effort. So, shortcomings of any choice being made show up too late for reversing the choice Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 9

Nokia Research Center d, the er are e, not the The testing tool chain Nokia Research Center d, the er are e, not the The testing tool chain , what are em on. But C A R/ M E M /V T T Run Time System Executable Test System (ETS) k Codec Platform Adapter m <-> Test lization of SUT Adapter SUT Test Development Tools Editor Compiler Codec Generator Testware Test cases … Test System Codec Router • for each stage components from different vendors need to interoperate • clearly defined interfaces are needed Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 10 Test Execution Control Log Analysis

Nokia Research Center The testing tool chain interfaces C A R/ M E M Nokia Research Center The testing tool chain interfaces C A R/ M E M /V T T Run Time System Executable Test System (ETS) TRI Platform Adapter TCI/Log Codec SUT Adapter SUT Test Development Tools Editor Compiler Codec Generator Testware Test cases … Test System Codec Router Test Execution Control Log Analysis • TTCN-3 core notation language for exchange of test cases • Run Time IF for Adaptors and Codecs • Test Execution Control and logging through TCI Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 11

Nokia Research Center Further work on interfaces Tool vendors have a notion to “close Nokia Research Center Further work on interfaces Tool vendors have a notion to “close shop” and not support public interfaces • Codec interface part in TCI leads to non-interchangeable codecs • • realisation of a given specification in IDL depends on the target language. • Java based tools and C based tools require different codecs • performance problems when separating the codec to a separate executable Standardisation sometimes lags behind (e. g. TCI) C A R/ M E M /V T T • • It is also desirable to generate as many test cases and parts of the environment as possible directly from the behavioural description of the IUT • -> modelling representations (e. g. XMI) need to be supported in the tools Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 12

Nokia Research Center C A R/ M E M /V T T Overview Variations Nokia Research Center C A R/ M E M /V T T Overview Variations in Test Systems TTCN-3 in existing test environments TTCN-3 for testing hardware Conclusion/Summary Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 13

Nokia Research Center Hw design flow C A R/ M E M /V T Nokia Research Center Hw design flow C A R/ M E M /V T T Algorithmic description informal model, e. g. C source code Decomposition (Block diagram) Highest Level behavioural model (e. g. System. C) Bit accurate Hardware model (e. g. VHDL) Notion of “Time” Notion of “Clock Cycles” Introduction of physics Physical Model (layout, transistor, …) (e. g. RTL, Spice, …) • Description consistent through all stages/levels • Automatic generation of tests (“Test vectors”) • Back annotation integral part of the process Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 14

Nokia Research Center Sw design flow use case description C A R/ M E Nokia Research Center Sw design flow use case description C A R/ M E M /V T T Decomposition formal model, e. g. UML UC (package diagram) Highest Level behavioural model (e. g. UML sequence diagram, state chart) Refinement, more detailed behavioural model Introduction of “Signals” (e. g. UML sequence diagram, state chart) Introduction of programming language syntax executable (C, C++, Java, …) (e. g. binary, script, …) • • • Rather “novel” approach Automatic generation of tests (“Test vectors”) still in experimental phase Back annotation is not formally specified Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 15

Nokia Research Center C A R/ M E M /V T T “Traditional” HW-SW Nokia Research Center C A R/ M E M /V T T “Traditional” HW-SW integration use case description Algorithmic description Decomposition formal model, e. g. UML UC informal model, e. g. C source code (package diagram) Decomposition (Block diagram) Highest Level behavioural model (e. g. System. C) (e. g. UML sequence diagram, state chart) Refinement, Bit accurate Hardware model Introduction of “Signals” more detailed behavioural model (e. g. VHDL) Notion of “Time” Notion of “Clock Cycles” (e. g. UML sequence diagram, state chart) Introduction of programming language syntax executable (C, C++, Java, …) Physical Model (e. g. RTL, Spice, …) Introduction of physics (layout, transistor, …) (e. g. binary, script, …) HW Prototype SW release HW prototype Integration Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 16 Integration Tests

Nokia Research Center C A R/ M E M /V T T Common nodes Nokia Research Center C A R/ M E M /V T T Common nodes in the HW and SW design flow • Integration tests are not determined by formal derivation from HW/SW high level models • Integration results feed back “somewhere” in the process in SW • HW and SW tests only derive from HW and SW models respectively How about • utilisation of SW models (e. g. High level behavioural) for generation of HW tests? • including behavioural HW models (System. C) in High level behavioural SW models? • generation of tests directly from the model representation? Integration typically is the 1 st stage where HW and SW meet • • Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 17

Nokia Research Center C A R/ M E M /V T T Overview Variations Nokia Research Center C A R/ M E M /V T T Overview Variations in Test Systems TTCN-3 in existing test environments TTCN-3 for testing hardware Conclusion/Summary Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 18

Nokia Research Center C A R/ M E M /V T T Summary/Conclusions • Nokia Research Center C A R/ M E M /V T T Summary/Conclusions • Further deployment of TTCN-3 needs convincing advantages: – integration into existing test environments • interfaces, replaceable components from heterogenous test systems – handling of variants through specific coding constructs – automated generation of test cases from abstract models – cover HW as well as SW through adequate representation in SW models Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 19

C A R/ M E M /V T T Nokia Research Center Thank you! C A R/ M E M /V T T Nokia Research Center Thank you! Questions? Acknowledgement: The authors would like to thank Mr. Risto Teittinen from Nokia Networks for his contributions to this topic Making TTCN-3 work © NOKIA Originator: Martin Botteck / April 12, 2005 / Page: 20