Скачать презентацию Automated software te sting Egidijus Bab e nskas Скачать презентацию Automated software te sting Egidijus Bab e nskas

3eed868340cf058c1e96e5f5de29e6d2.ppt

  • Количество слайдов: 36

Automated software te sting Egidijus Bab e nskas, IFM 0/2 1 Automated software te sting Egidijus Bab e nskas, IFM 0/2 1

Topics 0 Automated software testing. What? Why? 0 Automated vs. Manual 0 Common types Topics 0 Automated software testing. What? Why? 0 Automated vs. Manual 0 Common types of automated testing methods 0 Automated software testing framework 0 Summary 2

What is automated software testing? A process which covers: 0 the use of software What is automated software testing? A process which covers: 0 the use of software to control the execution of tests 0 the comparison of actual outcomes to predicted outcomes 0 the setting up of test preconditions 0 other test control and test reporting functions. 3

Why testing is critical? 4 Why testing is critical? 4

Why automated testing? 0 A time savings translates directly into cost savings 0 Improves Why automated testing? 0 A time savings translates directly into cost savings 0 Improves testing productivity 0 Improves accuracy 0 Increases test coverage 0 Does what manual testing cannot do 5

Effort and Cost 0 0 0 0 Let’s assume 6 test cases Effort required Effort and Cost 0 0 0 0 Let’s assume 6 test cases Effort required to run all 6 manually => 10 min Effort required to write unit tests for all 6 cases => 10 min Effort required to run unit tests for all 6 cases => < 1 min Number of testing iterations => 5 Total manual testing time => 50 min Total unit testing time => 10 min Release Manual Test Auto Test Manual Test Cumulative 1 10 10 10 2 10 0 20 3 10 0 30 4 10 0 40 5 10 0 50 6

Automated vs. Manual testing 0 Not all tests can be automated and most times Automated vs. Manual testing 0 Not all tests can be automated and most times is difficult to decide what to automate and what to manually test. 7

Advantages Automated Testing • Need to run a set of tests repeatedly • Helps Advantages Automated Testing • Need to run a set of tests repeatedly • Helps performing "compatibility testing" (on different configurations and platforms) • Long term costs are reduced Manual Testing • Test cases have to be run a small number of times • Allows the tester to perform more specific tests • Short term testing costs are reduced • Possible to run regressions on a • The more time tester spends code that is continuously testing a module the grater changing and in shorter time chance to find real bugs 8

Disadvantages & other factors Automated Testing • It's more expensive to automate (bigger initial Disadvantages & other factors Automated Testing • It's more expensive to automate (bigger initial investments) • You cannot automate everything, some tests still have to be done manually Manual Testing • Manual tests can be very time consuming • For every release you must rerun the same set of tests which can be tiresome Other Factors • The performance of your test tools • The knowledge level of your testing team • The continuous growth of software to be tested 9

Common types of automated testing methods 0 Monkey testing 0 Capture / playback 0 Common types of automated testing methods 0 Monkey testing 0 Capture / playback 0 Code-based (Unit) testing 0 Intelligent test automation 10

Monkey testing 0 Randomly selecting inputs from a large range of values and monitoring Monkey testing 0 Randomly selecting inputs from a large range of values and monitoring if exceptions are thrown. 0 For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input. 0 It applies not only for GUI or WEB testing, but also for Unit testing. 11

Tools example Monkey – Android testing tools. The simplest way to use the monkey Tools example Monkey – Android testing tools. The simplest way to use the monkey is with the following command, which will launch the application and send 500 pseudo-random events to it. $ adb shell monkey -v -p your. package. name 500 12

Capture / playback 0 It’s a set of software programs that capture user inputs Capture / playback 0 It’s a set of software programs that capture user inputs and stores it into a script for later replay. + Repeated testing can be performed quickly. + Does not require programming skills. - When the GUI changes, input sequences previously recorded may no longer be valid. - Difficult to determine location of bugs. 13

Tools example Autohotkey - Free keyboard macro program. Supports hotkeys for keyboard, mouse. 14 Tools example Autohotkey - Free keyboard macro program. Supports hotkeys for keyboard, mouse. 14

Code-based (Unit) testing 0 Individual units of source code are tested to determine if Code-based (Unit) testing 0 Individual units of source code are tested to determine if they fit for use. 0 Ideally, each test case is independent from the others + Bugs can be found in early development stage + Easy to test boundary cases - Not effective for the integrated system testing 15

Unit test-case generation method 0 Random 0 Genetic 0 Other… 16 Unit test-case generation method 0 Random 0 Genetic 0 Other… 16

Random 0 Input values generated randomly 0 Run with generated inputs 0 User checks Random 0 Input values generated randomly 0 Run with generated inputs 0 User checks output values (pass/fail) and saves them for later automatic checking (Oracle). 17

Genetic This methods approach is to combine JML (JAVA model language) and genetic algorithms. Genetic This methods approach is to combine JML (JAVA model language) and genetic algorithms. JUnit is used as a test execution platform. 18

Tools examples Parasoft Jtest is an automated Java testing and static code analysis product Tools examples Parasoft Jtest is an automated Java testing and static code analysis product Test. Gen is a collection of open-source tools that automatically generates unit test cases. 19

Intelligent test automation 0 A model-based method for generating tests from a description of Intelligent test automation 0 A model-based method for generating tests from a description of an application’s behavior (UML) + Models are simple to modify, so model-based tests are economical to maintain over the life of a project. - Most of these methods are researched only theoretically. 20

OCL + UML Inputs generator Constraints UML + OCL Tested software Oracle Outputs Constraints OCL + UML Inputs generator Constraints UML + OCL Tested software Oracle Outputs Constraints pass / fail Report 21

Automated testing framework 0 A framework is an integrated system that sets the rules Automated testing framework 0 A framework is an integrated system that sets the rules of automation of a specific product. 0 This system integrates the function libraries, test data sources, object details and various reusable modules. 0 The framework provides the basis of test automation and simplifies the automation effort. 22

Common types of frameworks 0 Data-driven testing 0 Keyword-driven testing 0 Hybrid testing 23 Common types of frameworks 0 Data-driven testing 0 Keyword-driven testing 0 Hybrid testing 23

Data-driven testing 0 Data-focused automation 0 User defines just data sets to run tests Data-driven testing 0 Data-focused automation 0 User defines just data sets to run tests with 0 Data is defined in external data source and decoupled from script 0 Good fit for a single application testing, featuring multi-environment, big datasets, and rarely changing Test Cases + - Good usability and reusability Quality and coverage strongly depends on the automation skills of implementing person Good test flow coverage Continuous maintenance issues due to a big amount of the code Multi-environment support Manual validation is required 24

Data set Example Test script 25 Data set Example Test script 25

Keyword-driven testing 0 Good fit for a multi-application testing featuring multi-environment, big datasets, and Keyword-driven testing 0 Good fit for a multi-application testing featuring multi-environment, big datasets, and a big number of short and straight Test Cases 0 Users de-compose test cases into reusable action keywords 0 Non-coders are able to author automated test cases with action keywords + - Test development doesn’t require Due to keywords limit doesn’t allow covering programming skills very complicated Test Cases Good test flow coverage Meta-language specific training is required for staff Compact code Manual validation is required 26

Example 27 Example 27

Hybrid testing 0 Combines the best of data-driven and keyword-driven testing: 0 User defines Hybrid testing 0 Combines the best of data-driven and keyword-driven testing: 0 User defines data sets to run tests with 0 User also defines flow control through action keywords 0 Data is separated from script and stored in external data source with action keywords + Test Plan / Test Scenario creation doesn’t require programming or training on the Testing Tool Compact code and scalable architecture Built-in strict validation Requires initial investment for design and implementation of the framework 28

Framework components 0 Test generation (data and script generators). Tools might create specialized data Framework components 0 Test generation (data and script generators). Tools might create specialized data such as randomized email messages, or populate databases etc. 0 System configuration. Tools might preserve or reproduce system parameters. 0 Test execution. Tools might operate the software itself, either simulating a user working through the GUI and using an alternative testable interface. 0 Oracles. An oracle is any mechanism by which we detect failure or success. Tools might automatically detect certain kinds of error conditions in a product. 0 Activity recording & coverage analysis. Tools might watch testing as it happens and retrospectively report what was and was not tested. 0 Test management. Tools might record test results, organize test ideas or metrics. 29

General framework structure 30 General framework structure 30

Common functional automated test tools Tool IBM / Rational Functional Tester (RFT) + • Common functional automated test tools Tool IBM / Rational Functional Tester (RFT) + • Built as Eclipse plug-in with full IDE, Java support and source management • Supports Web-based applications, . Net, Java, Siebel, SAP®, Power. Builder, AJAX, Adobe Flex, Adobe PDF documents • Full GUI Object Map repository - • License cost HP / Mercury • Supports Web 2. 0, GWT, Dojo, YUI, Quick Test Silverlight, Flex, Ajax, JAVA, . NET Pro (QTP) applications • Full GUI Object Map repository • Seamless integration with Quality. Center • No IDE (changing in release 10) • License cost • Visual. Basic scripting is limited Selenium RC & IDE • No GUI Object repository • Only web-based application support • Tool support is 31 limited • Good browser support • Good language support (JAVA, Ruby, C#) • Can be easily extended as JUnit suite • Open-source (no license cost)

Tools examples 32 Tools examples 32

Summary 0 Automated testing improves productivity, accuracy, increases test coverage, saves time and cost Summary 0 Automated testing improves productivity, accuracy, increases test coverage, saves time and cost 0 Need to decide what to automate and what to manually test 0 The framework provides the basis of test automation and simplifies the automation effort. 33

More information 0 http: //www. slideshare. net/lfingerman/test-automation-best-prcaticeswith-soa-test-approach 0 http: //opuntia. cs. utep. edu/utjml/utest. html More information 0 http: //www. slideshare. net/lfingerman/test-automation-best-prcaticeswith-soa-test-approach 0 http: //opuntia. cs. utep. edu/utjml/utest. html 0 http: //www. slideshare. net/yogindernath/understanding-of-automation -framework-presentation 0 http: //www. ibm. com/developerworks/rational/library/591. html 0 http: //www. satisfice. com/presentations/agileauto. pdf 0 http: //automation-beyond. com/2009/05/30/test-automation-practices -recordplayback/ 0 http: //en. wikipedia. org/wiki/Test_automation 0 http: //www. slideshare. net/directi/automated-testing-vs-manualtesting-391121 34

Thank you! 35 Thank you! 35

Question ? ? ? 0 What is the benefit of automated testing? 0 What Question ? ? ? 0 What is the benefit of automated testing? 0 What are the common types of automated testing methods? 0 What are the common types of frameworks? 0 What components come to framework? 0 What are the examples of automated testing tools? 36