Contributions to Automated Testing of Network Service Interactions Rudy Deca 1 , Omar Mahrez 1, Omar Cherkaoui 1, Yvon Savaria 2 and Doug Slone 3 1 University of Québec at Montréal, 2 Ecole Polytechnique de Montréal, 3 Cisco Systems, Inc. Automated Testing of Network Service Interactions Notere 2005, Ottawa/Gatineau, Canada
Contents The problem context ¡ The Meta-CLI Model ¡ The Script. Maker tool ¡ Conclusion Ø Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 2
The Problem context Causes: Connectionless stateless networks Connectionoriented circuit-switching stateful network services More services more features more parameters More environment states Effects: Network convergence Increase in the number of states Hybrid, contradictory network service characteristics New Solutions required More combinations more interactions more configurations more tests required 3
Contents ¡ Ø The problem context The Meta-CLI Model Ø General presentation Ø • l ¡ ¡ Structure Operations Abstracting the environment The Script. Maker tool Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 4
The Meta-CLI Model solution ¡ Abstracts: l l l ¡ the network service and device configuration states the environment states the dynamic changes of the connectionless, stateless IP protocols Benefits: l l Automated Testing of Network Service Interactions Automated adaptation to environment changes Efficient generation and modification of configurations, test scripts and tests for service interactions. R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 5
Modelling a service configuration as a conceptual tree structure service_1 device_1 Created Modified service_1 Meta-CLI modelling Command mode M Configuration commands and parameters Command P Parameter R Arcs: ¡ Hierarchical relationships ¡ Inclusion relationships Command mode N Command Q Parameter S Nodes: ¡ Configuration command modes ¡ Configuration commands ¡ Configuration parameters 6
Contents ¡ Ø The problem context The Meta-CLI Model Ø General presentation • Structure Ø Operations Ø Ø l ¡ ¡ Configuration Validation Abstracting the environment The Script. Maker tool Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 7
Meta-CLI Model Operations ¡ Operation types: l l ¡ Distribution: l l ¡ Configuration Validation Single device Multiple devices of a network Operation attributes: l l l Automated Testing of Network Service Interactions Atomicity Order (causal, sequential) Granularity (varying from parameters to services) R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 8
Configuring services using the Meta-CLI Model The main idea of the Meta-CLI configuration: compose the device and network service configuration models. Automated Testing of Network Service Interactions 9
Generic service configuration operations ¡ ¡ ¡ ¡ Instantiate service configurations on devices. Instantiate service configuration parameter values. Instantiate service configuration constraints. Adapt service configuration to device and interface environment. Adapt service configuration to device and interface role. Determine environment support for various feature combinations Determine compatibility of services and features. Determine environment support for service and feature combinations. Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 10
Examples of validation dependencies device_1 • Command • device_2 • Command P • Command Q • Value 1 • Command P T • Value 5 • Value 2 • Value 6 Dependency T-Q • Parameter R • Value 3 Dep. U-R U • Value 4 • Parameter S • Value 7 • Parameter V • Value 8 Dependency S-R Validation dependencies between: ¡ configuration commands: T depends on Q ¡ configuration parameters: S and U depend on R 11
Network service and feature interactions ¡ Mutual exclusion, due to: l l l ¡ Influence, due to : l l ¡ Competition for resources Accessing the same parameters/variables Incompatible goals Scale factor Performance factor Service combination constraints: l l Common supporting environment Existence of mandatory parameters Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 12
Contents ¡ ¡ The problem context The Meta-CLI Model • Ø ¡ ¡ General presentation Abstracting the environment The Script. Maker tool Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 13
Environment heterogeneity solution The configuration environment: ¡ network-level component l l ¡ network topology, technology, Protocol, Role. equipment-level component. l hardware components ¡ ¡ ¡ l software components ¡ ¡ Automated Testing of Network Service Interactions device type, interface type, medium type, etc. operating system, software version, etc. R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 14
Adapt CLI commands to different environments service_1 Command P is instantiated on device_1 with one parameter (R) and on device_2 with two parameters (R, S). env 1, env 2 Command P env 1, env 2 Paramet. R env 1, env 2 device_1 + service_1 Paramet. S env 2 device_2 + service_1 device_2 device_1 env 2 env 1 Command P Parameter R Parameter S 15
Adapt CLI commands to different device and interface roles service_1 role 1, role 2 Command P is instantiated: ¡ on device_1 with parameter R and ¡ on device_2 with parameter S, based on the different roles assumed by the two devices. Paramet. R role 1 • device_1 role 1 Paramet. S role 2 + service_1 device_2 role 2 Command P Parameter R Parameter S 16
Addition of a network service to a device configuration ¡ ¡ ¡ The generic network service model is accessed from the repository. A service instance model is generated and adapted to the given environment and then composed with the device configuration model. 17
Modification of a network service in a device configuration ¡ ¡ The network service is already configured on an equipment when an environment change occurs. The network service model instance in the old environment is extracted from the device configuration model. The generic network service model is accessed from the repository. A new instance model is l l generated for the new environment and composed with the device configuration model.
Determining environment support for various feature combinations service_1 Service features env 1, env 2 Command P env 1, env 2 Feature R Feature S env 1, env 2 Environment instances P P Feature R NP P Feature S service_1 NP NP P P env 1 S S NS NS env 2 S S P = is present S = supports NP = is not present NS = does not support The information is obtained by querying the tree model. Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 19
Contents ¡ ¡ Ø The problem context The Meta-CLI Model The Script. Maker tool Ø • • • ¡ Functionality Architecture Generation scenario Conversion scenario Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 20
Functionality of the Script. Maker tool ¡ ¡ ¡ Aimed to regression testing. Provides an effective, automated and easy way to generate/modify testing scripts with test cases featuring various combinations of network services and features. Solves the problem of the existence of a large number of : l l l ¡ combinations among existing network services; features and parameters of the services; environment states in which the service parameters are configured differently. Uses a modular approach for components: l l Service configurations Traffic Routing, topology and connection among router interfaces Configurations validation Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 21
Contents ¡ ¡ Ø The problem context The Meta-CLI Model The Script. Maker tool • Ø • • ¡ Functionality Architecture Generation scenario Conversion scenario Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 22
Architecture of the Script. Maker tool Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 23
Contents ¡ ¡ Ø The problem context The Meta-CLI Model The Script. Maker tool • • Ø • ¡ Functionality Architecture Generation scenario Conversion scenario Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 24
The script generation scenario Script Manager Block Environme nt (descripto Feature r) name(s) OK Script Feature name(s) Environment (descr. ) Signaling Selected Test Selection Generic TC, TS, Block Test Filling Block Selected Specific TC, TS, Test Binding Block Templates Generic service capsule stubs Automated Testing of Network Service Interactions Envi Ron men Meta-CLI t Templates Specific CLI service code Model Block 25
The test selection block Test Selector Feature name(s) TC, TS, Templa te names Spec Archive Featur e name( s) Selected TC, TS, Templates Env. Constr Selected desc. TC, TS, Templat e names Test Organizer Envi -ron. Feature men name(s) t Test Bench Constrain ts Environment Constraint Database Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 26
The test filling block Test Filler Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology commands Selected generic stubs Kx, Kv, Trf, Top Converter Selected Gen stubs Kx, Kv, Trf, Top Code Generator Automated Testing of Network Service Interactions Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology commands Environment Selected Gen CLI auxiliary, validation commands and TCL Traffic, Topology commands Instantiator commands R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone Ks = Service capsule Kv = Validation capsule 27 Kx = Auxiliary capsule
The Meta-CLI Model block Selected generic service stubs Ks Meta-CLI Model manager Selected generic service capsule Service Retriever Selected generic service stubs Selected generic service capsule Capsules (models) Automated Testing of Network Service Interactions Selected specific CLI service Environment descriptor Selected generic service capsule Service Instantiator R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 28
Contents ¡ ¡ Ø The problem context The Meta-CLI Model The Script. Maker tool • • • Ø ¡ Functionality Architecture Generation scenario Conversion scenario Conclusion Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 29
The script conversion scenario Environmen t Feature name(s) Old script Script Manager Block New script Script Feature name(s) Parsing Block Script Environme nt Generic TC, TS, Template services auxiliaries validations topologies traffics Automated Testing of Network Service Interactions Test Selection Block Signaling Selected Generic TC, TS, Template selected services auxiliaries validations topologies traffics service Test Filling Block Envi Ron men t Meta-CLI Filled Selected Specific TC, TS, Template Test Binding Block Specific CLI service code Model Block 30
Conclusion ¡ Problem: l New Internet services, hybrid, with numerous: ¡ ¡ l ¡ parameters features interactions heterogeneous environments. FSMs not always useful. Solutions: l The Meta-CLI Model: ¡ ¡ l Generic service configuration Environment-aware The Script. Maker tool: ¡ ¡ ¡ Automated Testing of Network Service Interactions Automated generation/modification of testing scripts; Modular design; Uses the Meta-CLI for abstracting and reusing service configurations. R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 31
Questions Automated Testing of Network Service Interactions R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone 32
Скачать презентацию Contributions to Automated Testing of Network Service Interactions
a510c33a9401c94cf65c549277ab774c.ppt