Selecting GUI Automation Testing Tool for Mobile Domain

Selecting GUI Automation Testing Tool for Mobile Domain Mohit Anand, Software Engineer Adobe 1

Abstract With the growing number of mobile apps and their agile development life cycle, Automation testing around them will no longer be optional. The intent of this paper is to present a comparison between the widely used mobile automation tools in order to determine their ease of use & effectiveness. We picked up a typical Adobe mobile app, tested its GUI manually & then followed it up by a round of automation testing using these tools. This paper list down the technical nuances involved while choosing any such tool for user specific scenario. 2

Do you know your apps? 3

Complexities in Mobile Landscape Operating System Screen size Screen resolution Device Manufacturers Smartphone or Tablet GPU used 4

Operating System & Device Manufacturers 5

Screen Size & Resolution 6

GPU: Heart of Creative Apps 7

Localization: The Big hurdle Globalizatio n ready Mobile apps stores are global, Mobile apps should be too. Supporting the app in around 18 locales 8

Our main consideration GUI Testing Functional Testing Appealing Design Positive interaction Usability App success 9

Need of Automation in Mobile Testing Agile development cycle Mobile Landscape Complexities Globalization: Apps releasing in multiple languages Automation is the key Growing mobile apps 7(Q 1’ 14) – 40(Q 1’ 16) 10

Mobile Automation Tools Monkey. Runner UI Automator 11

Selection criteria We want to have a combination of tools which satisfy the following criteria: v. Can automate Android/i. OS/Both mobile apps v. First party/Third party tool v. Bundled with the OS IDE/ Should be installed separately v. Coordinate based/ Object based automation 12

Tools Selected Monkey. Runner UI Automator 13

Monkey. Runner Key Features • Provides an API for writing programs to control an Android device/emulator. Google • Can be used for both functional and UI testing. • Entirely based on coordinates based automation. Components of this tool • Monkey. Runner • Monkey. Device • Monkey. Image 14

Monkey. Runner Strengths Easy to set-up. Test multiple devices at a time. Appropriate for regression testing of a basic app. Random testing. Limitations Only supports Android. Different script for different device. Even slight change in UI breaks the script. 15

UI Automator Key Features • It is a java library containing API’s to create customized functional UI tests. Google • Integrated with Google’s own Android development IDE • Entirely based on object based automation. Components of this tool • Uiautomatorviewer • API framework. 16

UI Automator Strengths Limitations One of the best tools for testing android apps. Only supports Android (versions > 4. 3). Comes bundled with Android Studio IDE. Lack of support for Web view. Appropriate for UI testing of any android app. Lack methods for some gestures like long press on an element. 17

Key Features • It is an integrated environment for recording, customizing and managing test suites. Cloud Monkey • Open source, eclipse based IDE. • Used for automation of both i. OS & Android apps. • Limited features are available in the free version of this tool. Paid version is costly. Components of this tool • Monkey. Talk IDE • Monkey. Talk Agent 18

Strengths Limitations Supports both i. OS and Android apps. The essential features are available in the paid version. Test multiple devices at a time. Doesn’t support web view. Open source Agent needs to be installed in the app which needs to be tested. Image comparison is supported Limited x. Path support. 19

Key Features • It is a HTTP server that takes HTTP commands from the test script and executes the command on the device. Sauce Labs • Based on Web. Driver JSON protocol. • Used for automation of both i. OS & Android apps. • Totally derived from selenium code + mobile gesture handling Components of this tool • Appium Server • Appium Client libraries • Appium Inspector 20

Strengths Limitations Supports both i. OS and Android apps. No support for parallel test for i. OS apps. One of the most versatile tools for testing any native, hybrid and mobile web apps. No support for Android version < 4. 1 (selendroid can be used) Doesn’t require code access or recompiling the app. No Image comparision. Test multiple android devices at a time. No option to switch off network connection. 21

Deep dive into Appium 22

Test Setup 23

The Process: Recipe of Success Know your mobile app. Identify the right automation tool. Write test script to cover all the testing workflows in the app. Run the script for the test pass. Next Release: Update the script. Repeat until the app lives on the app store. 24

Demo of Adobe Shape GUI Automation 25

Takeaways v Use real devices: In Adobe, we say no to simulators. v All elements in the UI should have a unique resource id. v. The App UI has changed: Get ready for modifying your test script. v. Say NO to sleeps and x. Path. vi. OS Automation: You need code access or debug build. 26

Choose the RIGHT tool Preference to free/paid tool Which app to automate: Android/i. OS or both? First party/Third Party RIGHT TOOL Community support Time Crunch Flexibility of tester with the scripting languages 27

Use case scenario Android app only Basic UI testing Time Crunch Low effort/investment Monkey Runner Android app only Intensive UI testing. Don’t want to use third party tool. Have time to build a stable automation solution. UI Automator Android & i. OS app Your app doesn’t supports web view. Image comparison requirement. Can install agents in your app / can invest money per month Monkey Talk Android & i. OS app Intensive UI testing Don’t have code / Don’t want to install any agent. Stable automation solution (free) for suite of apps Appium 28

Bibliography o http: //developer. android. com/tools/help/monkeyrunner_concepts. html o https: //www. cloudmonkeymobile. com/monkeytalk o http: //developer. android. com/training/testing/ui-testing/index. html o https: //developer. android. com/about/dashboards/index. html o https: //developer. apple. com/support/app-store/ 29

Question and Answers 30

31