#22 Automation Testing Tools

What is Test Automation
 Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.

Why do we need test automation
 Run existing tests on a new version of a program
 A lot of programs are frequently always modified in the same environment
 Run more tests more often
 Giving possibilities to run more tests which are run more often
 Perform tests that would be difficult or impossible to do manually
 Better use of resources
 Saving time
 Better use of resources
 Testers can do better jobs, machines can be run all the time.

Benefits of automated testing
a. Manual testing
 Time-consuming
 Tedious
 Depend on tester’s emotions => Serious defects may be undetected

b. Automated Testing
 Fast
 Reliable
 Repeatable
 Programmable
 Comprehensive
 Reusable

Test Tools
 Application test tools
 Source test tools (depends on program language)
 Functional test tools
 Performance test tools
 Embedded test tools
 Web test tools
 Functional test tools
 Performance test tools
 Test management tools
 Bug tracking tools

Test Tools Comparison
 Major features in Automated test tools
 Record & play back
 Web Testing
 Data function
 Image testing
 Extensible language
 Environment support
 Tools in comparison
 Rational Robot
 Ruby
 OpenSTA
 Quick Test Pro
 …

Record & Playback
 Details how easy it is to record & playback a test.
 Does the tool support low-level recording (mouse drags, exact screen location)
 Is there object recognition when recording and playing back or does it appear to record ok but then on playback (without environment change or unique id’s, etc changes) fail?
 How easy is it to read the recorded script

Web Testing
 Are there functions to tell me when the page has finished loading?
 Can I tell the test tool to wait until an image appears?
 Can I test whether links are valid or not?
 Can I test web based objects functions like is it enabled, does it contain data, etc
 Are there facilities that will allow me to programmatically look for objects of a certain type on a web page or locate a specific object?
 Can I extract data from the web page itself? E.g. the title? A hidden form element?

Data Function
 Does the tool allow you to specify the type of data you want?
 Can you automatically generate data?
 Can you interface with files, spreadsheets, etc to create, extract data?
 Can you randomise the access to that data?
 Is the data access truly random?

Image Testing
 Does the tool provide OCR (optical character recognition)?
 Can it compare one image against another?
 How fast does the compare take?
 If the compare fails how long does that take?
 Does the tool allow you to mask certain areas of the screen when comparing?

Type of Automation Test
 Functional Testing
 Performance Testing

Performance Testing Features (1)
 Performance Test Recording
 Flexible User Scenarios: To emulate the load test of realuser activities, an easy, complete and customizable building of user scenarios are provided.
 Real-world Performance Testing: Real-world web load testing allows you to dynamically vary the number of virtual users hitting your web applications/web sites to study the load testing/stress testing conditions under varying load.
 Dynamic Data Replacement: Modifying the performance test scripts to reflect dynamic data requires no programming skills.

Performance Testing Features (2)
 Server Performance Monitors: Load test your web sites/web applications with integrated monitoring of server machine's system resources (cpu % and memory usage).
 Configurable Playback Options: Wide range of play options that affects the playback of load test scripts.
 Proxy Support: Peformance testing can be done through proxy servers, supporting authentication and proxy exclusions for local networks.
 Distributed Load Testing: Option to simulate thousands of simultaneous users working from a single machine or distributed on multiple machines. Centralized coordination and reporting of distributed load testing results.

Performance Testing Features (3)
 Datapools for Parameterization: To use unique data for each user in load testing/stress testing
 Configurable Think Time: Configurable think times to perform real-world performance testing where each user spends thinking time (wait time) before performing the next action in the web page.
 Browser Simulation: Support for simulating exact browser behavior (MSIE, Firefox and Mozilla simulation).
 Random Delay: Random Delay for Virtual user start to simulate user visit to the web site/web application in irregular intervals.
 Bandwidth Simulation: Option to emulate different network speeds during playback.

Functional Testing Features (1)
 Playback Options
 Automatic/Customizable Error Recovery for unattended testing to handle unexpected window, like an ASSERT box, pop- ups, etc.
 Playback Synchronization to handle the variation in time the application takes to load a new page.
 Option to chain scripts for controlling the order of script execution.
 Multiple Playback Options that enables testers to debug test errors while creating and maintaining test scripts.
 Allows execution of individual and/or groups of tests from one or many workstations.

Functional Testing Features (2)
 Scripting Capabilities
 Simplified Script Creation with recording provision.
 Records parent/child frames, PopUp's, modal and modeless dialogs.
 Learn mode to learn objects into the object repository.
 Keyword-driven testing allows automation experts to have full access to the underlying object repository to author and debug scripts using pre-defined keywords.

Functional Testing Features (3)
 Scripting Capabilities …
 Standard Scripting Language allows you to easily create/update scripts without much programming knowledge.
 Object Repository eliminates the need to modify all the scripts each time the application is modified.
 Data-driven Testing enables you to perform functional testing in different scenarios by just changing the test data in an external data source.
 Unicode Support allows you to test multi-language deployments of your applications..


Functional Testing Features (4)
 Portability
 Allows you to record scripts in Windows and replay it in Linux without re-creating the scripts.
 Browser Abstraction Layer allows scripts recorded in one browser to be replayed in all other supported browsers.
 Runtime locale option lets you to simultaneously test all language versions of your application with a single script.

Functional Testing Features (5)
 Reporting Capabilities
 Clear and Powerful Reports are provided to indicate the status of the test execution.
 Hyperlinks allow easy navigation through the report. This helps you to quickly identify application failures and clearly assess application quality.
 Supported Environments
 Internet Explorer 6, Firefox 1.5.x and Mozilla 1.7.x, Windows XP, 2000 and Linux …