Friday, 28 November 2014

Generic Test Automation Framework Design

Test Automation Framework Design

When test automation is taken to highest level tests can be started with a push of a button, they can be left running over-night unattended and next morning test results are published. This kind of automation clearly requires some kind of a system which makes creating, executing and maintaining test cases easy. The system should provide some core functionality (e.g. monitoring and reporting) and allow extending itself to make it possible to create new kinds of tests.

Characteristics of a good Test Automation Framework

1)      The framework MUST execute test cases automatically. That includes also for example verifying results, handling errors and reporting results.
2)      The framework MUST be easy to use without programming skills.
3)      The framework MUST be easily maintainable.
4)      The framework MUST be able to execute tests unattended.
5)      It MUST be possible to start and stop test execution manually.
6)      It SHOULD be possible to start test execution automatically at predefined time.
7)      Non-fatal errors caused by the SUT or the test environment MUST be handled gracefully without stopping test execution.
8)      Every executed test case MUST be assigned either Pass or Fail status and failed test cases SHOULD have a short error message explaining the failure.
9)      Test execution MUST be logged.
10)   Test report MUST be created automatically. Test report SHOULD be published automatically.
11)   The framework MUST have coding and naming conventions.
12)   The framework MUST be adequately documented.

High level Framework design

Components of a Automation Framework Architecture

Test Design System

Test design system is used for creating new test cases and editing existing ones. It is test engineers’ and domain experts’ playground and must be easy to use with minimal training and without programming skills.

The created test data can be stored into flat files or databases. A simple solution is using an existing tool like spreadsheet program or HTML editor as a test design system. A special test design tool made just for this purpose would of course have its benefits but building something like that is out of the reach for most automation projects.

 Test Monitoring System

Test monitoring system is used for controlling test execution and checking test results.
It should have at least the capabilities listed below.
• Starting test execution manually.
• Stopping test execution.
• Monitoring test execution while tests are running.
• Viewing test logs while tests are running and afterwards.
• Viewing test report.

Test Execution System

Test execution system is the core of the framework. Its main components are
a)      driver scripts,
b)      the test library,
c)        the test data parser and
d)      other utilities like logger and report generator

a)      Driver Scripts
Test execution is controlled by driver scripts which are started using the test monitoring system. Driver scripts are pretty short and simple because they mainly use services provided by test libraries and other reusable modules. 

b)      Test Libraries
Test libraries contain all those reusable functions and modules which are used in testing or in activities supporting it. Testing is mainly interacting with the tested system and checking that it behaves correctly. Supporting activities are often related to setting up the test environment and include tasks like copying files, initializing databases and installing software. Functions in the test library can do their tasks independently but they can also use other functions or even external tools.

c)       Test data parser
The test data parser  gets test data easily to driver scripts. Its task is processing the test data and returning it to the driver script in easy to use test data containers. The role of the parser may seem small but in reality it is the heart of the whole test execution system. 

d)      Logger
An easy and modular way to fulfill them is having a common logger component with the required features. Both the driver script and test functions use the same logger which in turn writes messages to the common test log.

e)      Report Generator
The reporting requirements of the framework can be summarized saying that a concise report must be created and published.

Working of a Data Driven Framework

Simple test scripts have test data embedded into them. This leads to a problem that when test data needs to be updated actual script code must be changed .This kind of reuse is also problematic because one particular in the tested system may require updating all scripts. Because of these problems embedding test data into scripts is clearly not a viable solution when building larger test automation frameworks. A better approach is reading the test data from external data sources and executing test based on it. This approach is called data-driven testing.

Step 1: Editing and Storing Test Data

Because data-driven test data is tabular it is natural to use spreadsheet programs to edit it. Another benefit is that spreadsheet programs are often used for simple test management tasks and in that case there is less need for storing same data in different places or copying it back-and-forth.

Step 2: Processing Test Data

Implementing a script for parsing data-driven test data can be surprisingly easy with modern scripting languages. Test data is first read into data variable and then split to lines so that header row is excluded. Data lines are then processed one by one. Lines are split to cells from tabulator character and cells’ data assigned to variables making test data available in the script for actual testing.

Step 3: Test Execution

The test scripts written are invoked from the data sheet and then executed. The test scripts will in short act on the application under test and execute the test cases.

Step 4: Logging and Reporting

The logger will log the data in a file after each step of execution and after the test is complete, the test reporting mechanism, will generate reports with status.

Wednesday, 26 November 2014

User defined Generic Library functions in Selenium Web Driver Framework

Utility or User defined Generic Library functions in Selenium Web Driver Framework

Any framework must be reusable, if any functionality needs to be repeatedly used such functions can be written in 1 utility file and called multiple times from any other testcase. They are also called generic functions.

It doesn’t make sense in making the code redundant. A better approach is to create a utility file and keep all reusable functions in one place and use as per requirement.

User defined functions (UDF) are the functions which are created by users as per their own requirement. In order to use a UDF, it is necessary to call the appropriate package file at the beginning of the program.

Benefits of User Defined Functions

·         - Can be used in a number of places without rewriting the code.
·         - Code can be made less complex and easier to write.
·         - Parameters can be passed to the function.
·         - Simpler to invoke.
·         - Cleaner and more understandable code

For example:  It is a three steps process to open a URL. First Instantiate a New driver, second Apply an Implicit wait on the driver and third Navigate to URL.  Browser can be any browser; it can be Mozilla, IE or any. It makes sense to create a function for opening a browser which will accept an argument (Browser Type) and it will open that particular browser. This ‘Browser Type’ argument will be driven from separate config file. To achieve this few more functions are required.

Here instead of writing the entire code of finding element, we can write it in a simpler form using a generic function getObject().

Example code looks like this

We can rewrite it like

The code is much simpler to understand and as it uses generic function getObject(). Here “demo_yes_btn” is the object created at Object repository.

Code for function getObject() is as below :

public static WebElement getObject(String xpathKey) {
                  return driver.findElement(By.xpath(TestObjectRepository.getProperty(xpathKey)));
            }catch(Throwable t){
                  // report error
                  return null;

Steps to generic function Utility

    1.       Create a package in your folder structure and name it as “LibraryFunctions”.
    2.       Now create a class under the package called as “LibraryFunc”.
    3.       Now write all the methods, which you think might be re-used in you test cases. Example you could include, the following :
a)      Initialization of browser
b)      Finding objects
c)       Taking screenshots
d)      Handling alerts and pop ups etc

    4.       Now write the code for getObject() given above.
 public static WebElement getObject(String xpathKey) {
                  return driver.findElement(By.xpath(TestObjectRepository.getProperty(xpathKey)));
            }catch(Throwable t){
                  // report error
                  return null;
    5.       Now if you want to use this getObject() Function in any of the test cases, just import the LibraryFunc java and use  the function inside it.

4.       Now create a test case , which uses the Library function and reuses it.
Sample code :
 package TestPackage;

import java.util.Properties;

import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;

import LibraryFunctions.LibraryFunc;

public class testlibfunc extends LibraryFunc{

      public void testlib()
      //used instead of
     // driver.findElement(By.xpath("//*[@id='yes']")).click();



   7.       Similarly , Generic functions can be written for initializing browser, taking screen shots, handling alerts etc.