
3eed868340cf058c1e96e5f5de29e6d2.ppt
- Количество слайдов: 36
Automated software te sting Egidijus Bab e nskas, IFM 0/2 1
Topics 0 Automated software testing. What? Why? 0 Automated vs. Manual 0 Common types of automated testing methods 0 Automated software testing framework 0 Summary 2
What is automated software testing? A process which covers: 0 the use of software to control the execution of tests 0 the comparison of actual outcomes to predicted outcomes 0 the setting up of test preconditions 0 other test control and test reporting functions. 3
Why testing is critical? 4
Why automated testing? 0 A time savings translates directly into cost savings 0 Improves testing productivity 0 Improves accuracy 0 Increases test coverage 0 Does what manual testing cannot do 5
Effort and Cost 0 0 0 0 Let’s assume 6 test cases Effort required to run all 6 manually => 10 min Effort required to write unit tests for all 6 cases => 10 min Effort required to run unit tests for all 6 cases => < 1 min Number of testing iterations => 5 Total manual testing time => 50 min Total unit testing time => 10 min Release Manual Test Auto Test Manual Test Cumulative 1 10 10 10 2 10 0 20 3 10 0 30 4 10 0 40 5 10 0 50 6
Automated vs. Manual testing 0 Not all tests can be automated and most times is difficult to decide what to automate and what to manually test. 7
Advantages Automated Testing • Need to run a set of tests repeatedly • Helps performing "compatibility testing" (on different configurations and platforms) • Long term costs are reduced Manual Testing • Test cases have to be run a small number of times • Allows the tester to perform more specific tests • Short term testing costs are reduced • Possible to run regressions on a • The more time tester spends code that is continuously testing a module the grater changing and in shorter time chance to find real bugs 8
Disadvantages & other factors Automated Testing • It's more expensive to automate (bigger initial investments) • You cannot automate everything, some tests still have to be done manually Manual Testing • Manual tests can be very time consuming • For every release you must rerun the same set of tests which can be tiresome Other Factors • The performance of your test tools • The knowledge level of your testing team • The continuous growth of software to be tested 9
Common types of automated testing methods 0 Monkey testing 0 Capture / playback 0 Code-based (Unit) testing 0 Intelligent test automation 10
Monkey testing 0 Randomly selecting inputs from a large range of values and monitoring if exceptions are thrown. 0 For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input. 0 It applies not only for GUI or WEB testing, but also for Unit testing. 11
Tools example Monkey – Android testing tools. The simplest way to use the monkey is with the following command, which will launch the application and send 500 pseudo-random events to it. $ adb shell monkey -v -p your. package. name 500 12
Capture / playback 0 It’s a set of software programs that capture user inputs and stores it into a script for later replay. + Repeated testing can be performed quickly. + Does not require programming skills. - When the GUI changes, input sequences previously recorded may no longer be valid. - Difficult to determine location of bugs. 13
Tools example Autohotkey - Free keyboard macro program. Supports hotkeys for keyboard, mouse. 14
Code-based (Unit) testing 0 Individual units of source code are tested to determine if they fit for use. 0 Ideally, each test case is independent from the others + Bugs can be found in early development stage + Easy to test boundary cases - Not effective for the integrated system testing 15
Unit test-case generation method 0 Random 0 Genetic 0 Other… 16
Random 0 Input values generated randomly 0 Run with generated inputs 0 User checks output values (pass/fail) and saves them for later automatic checking (Oracle). 17
Genetic This methods approach is to combine JML (JAVA model language) and genetic algorithms. JUnit is used as a test execution platform. 18
Tools examples Parasoft Jtest is an automated Java testing and static code analysis product Test. Gen is a collection of open-source tools that automatically generates unit test cases. 19
Intelligent test automation 0 A model-based method for generating tests from a description of an application’s behavior (UML) + Models are simple to modify, so model-based tests are economical to maintain over the life of a project. - Most of these methods are researched only theoretically. 20
OCL + UML Inputs generator Constraints UML + OCL Tested software Oracle Outputs Constraints pass / fail Report 21
Automated testing framework 0 A framework is an integrated system that sets the rules of automation of a specific product. 0 This system integrates the function libraries, test data sources, object details and various reusable modules. 0 The framework provides the basis of test automation and simplifies the automation effort. 22
Common types of frameworks 0 Data-driven testing 0 Keyword-driven testing 0 Hybrid testing 23
Data-driven testing 0 Data-focused automation 0 User defines just data sets to run tests with 0 Data is defined in external data source and decoupled from script 0 Good fit for a single application testing, featuring multi-environment, big datasets, and rarely changing Test Cases + - Good usability and reusability Quality and coverage strongly depends on the automation skills of implementing person Good test flow coverage Continuous maintenance issues due to a big amount of the code Multi-environment support Manual validation is required 24
Data set Example Test script 25
Keyword-driven testing 0 Good fit for a multi-application testing featuring multi-environment, big datasets, and a big number of short and straight Test Cases 0 Users de-compose test cases into reusable action keywords 0 Non-coders are able to author automated test cases with action keywords + - Test development doesn’t require Due to keywords limit doesn’t allow covering programming skills very complicated Test Cases Good test flow coverage Meta-language specific training is required for staff Compact code Manual validation is required 26
Example 27
Hybrid testing 0 Combines the best of data-driven and keyword-driven testing: 0 User defines data sets to run tests with 0 User also defines flow control through action keywords 0 Data is separated from script and stored in external data source with action keywords + Test Plan / Test Scenario creation doesn’t require programming or training on the Testing Tool Compact code and scalable architecture Built-in strict validation Requires initial investment for design and implementation of the framework 28
Framework components 0 Test generation (data and script generators). Tools might create specialized data such as randomized email messages, or populate databases etc. 0 System configuration. Tools might preserve or reproduce system parameters. 0 Test execution. Tools might operate the software itself, either simulating a user working through the GUI and using an alternative testable interface. 0 Oracles. An oracle is any mechanism by which we detect failure or success. Tools might automatically detect certain kinds of error conditions in a product. 0 Activity recording & coverage analysis. Tools might watch testing as it happens and retrospectively report what was and was not tested. 0 Test management. Tools might record test results, organize test ideas or metrics. 29
General framework structure 30
Common functional automated test tools Tool IBM / Rational Functional Tester (RFT) + • Built as Eclipse plug-in with full IDE, Java support and source management • Supports Web-based applications, . Net, Java, Siebel, SAP®, Power. Builder, AJAX, Adobe Flex, Adobe PDF documents • Full GUI Object Map repository - • License cost HP / Mercury • Supports Web 2. 0, GWT, Dojo, YUI, Quick Test Silverlight, Flex, Ajax, JAVA, . NET Pro (QTP) applications • Full GUI Object Map repository • Seamless integration with Quality. Center • No IDE (changing in release 10) • License cost • Visual. Basic scripting is limited Selenium RC & IDE • No GUI Object repository • Only web-based application support • Tool support is 31 limited • Good browser support • Good language support (JAVA, Ruby, C#) • Can be easily extended as JUnit suite • Open-source (no license cost)
Tools examples 32
Summary 0 Automated testing improves productivity, accuracy, increases test coverage, saves time and cost 0 Need to decide what to automate and what to manually test 0 The framework provides the basis of test automation and simplifies the automation effort. 33
More information 0 http: //www. slideshare. net/lfingerman/test-automation-best-prcaticeswith-soa-test-approach 0 http: //opuntia. cs. utep. edu/utjml/utest. html 0 http: //www. slideshare. net/yogindernath/understanding-of-automation -framework-presentation 0 http: //www. ibm. com/developerworks/rational/library/591. html 0 http: //www. satisfice. com/presentations/agileauto. pdf 0 http: //automation-beyond. com/2009/05/30/test-automation-practices -recordplayback/ 0 http: //en. wikipedia. org/wiki/Test_automation 0 http: //www. slideshare. net/directi/automated-testing-vs-manualtesting-391121 34
Thank you! 35
Question ? ? ? 0 What is the benefit of automated testing? 0 What are the common types of automated testing methods? 0 What are the common types of frameworks? 0 What components come to framework? 0 What are the examples of automated testing tools? 36
3eed868340cf058c1e96e5f5de29e6d2.ppt