Learn TestNG:

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use. Junit and TestNG differ in the core design. Junit is a unit testing framework while TestNG addresses testing at a higher level. 

Let's discuss the three main differences between TestNG and Junit. 

1. Dependency Test: In Junit, one test case failure can cause a bunch of test cases to fail in the test suite. There is no option of skipping the set of dependent test cases. The dependent test cases are also reported as failures. For example, suppose there is a test case to test login and the next 10 test cases need to perform a transaction after login. If the login test case fails the other 10 test cases will also fail. TestNG handles dependency between test cases. If one test case failure causes the failure of a group of test cases it skips that group and executes the rest of the test suite. The group that has dependency on the failed test cases is reported as skipped NOT failed.
2. Groups: In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific subset of the test-suite say database related test cases instead of running the entire test suite. In Junit for a long time it was not possible to run a specific subset of the test cases. We can either run the entire suite or run each test case individually. Junit 4.8 introduced a new feature called “Categories” to overcome this limitation. However groups are much easier to configure in TestNG.
3. Parameterization: TestNG supports parameterization for objects. For example I can execute a single test case for multiple test data sets through the parameterization of DataProvider object. This makes the implementation of data driven testing more flexible in TestNG.

Annotations were formally added to the Java language in JDK 5 and TestNG made the choice to use annotations to annotate test classes.

Here is the list of annotations that TestNG supports:

Annotation	Description
@BeforeSuite	The annotated method will be run only once before all tests in this suite have run.
@AfterSuite	The annotated method will be run only once after all tests in this suite have run.
@BeforeClass	The annotated method will be run only once before the first test method in the current class is invoked.
@AfterClass	The annotated method will be run only once after all the test methods in the current class have been run.
@BeforeTest	The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
@AfterTest	The annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.
@BeforeGroups	The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.
@AfterGroups	The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.
@BeforeMethod	The annotated method will be run before each test method.
@AfterMethod	The annotated method will be run after each test method.
@DataProvider	Marks a method as supplying data for a test method. The annotated method must return an Object[ ][ ] where each Object[ ] can be assigned the parameter list of the test method. The @Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.
@Factory	Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return Object[ ].
@Listeners	Defines listeners on a test class.
@Parameters	Describes how to pass parameters to a @Test method.
@Test	Marks a class or a method as part of the test.

Let's see how we can use above annotations in out automated tests.

In order to continue further one should setup the TestNG plugin and add the TestNG jar file to the project build path in eclipse. If you don't know how to setup please follow the tutorial here - Selenium Webdriver and TestNG setup in Eclipse. You don't need to setup Selenium Webdriver at this stage, please follow instructions only related to TestNG.

Please follow the steps below to continue learning TestNG annotations.

• Create a new project and a class to it by following Step by step guide to setup Webdriver project in Eclipse  in the above post. But change the project, package and class names accordingly.
• Copy and paste the content below in to your new class.
• Right click on the class file and run it as TestNG Test and look at the Eclipse Java console. This looks like below,
• You have 3 tests in the class or in the test suite. @BeforeMethod and @AfterMethod have run 3 times before and after of every test case. These methods are called as configuration methods in TestNG. Generally in Selenium automation testing they are used for opening the browser before test and closing the browser after the test. And also you can you use it for opening and closing the resources.
• @BeforeClass and @AfterClass are run only once for all the tests in that class. 
• @BeforeSuite and @AfterSuite are run only once because they only run once per suite. A suite can have any no. of tests and classes, but these methods only runs per suite.
• So, all of these annotations can be used accordingly as per your needs in your test suite.
• At the beginning of the console logging you would have noticed a path to the xml file(C:\Users\username\AppData\Local\Temp\testng-eclipse-1346797963\testng-customsuite.xml). TestNG allows you to run your tests in several ways. TestNG xml is a XML file that describes the run-time definition of a test suite. It describes complex test definition while still remain easy to edit. 
• Copy the xml file from the path given in console and paste in your project root directory. This should look like below screenshot, 
  
• Open the xml file in eclipse and check how your test project is described in xml file.
• Now right click on xml file and run as TestNG Test. You will get the same results as before.
• TestNG also provides the default html report called 'test-output' for the test execution. This can be found in your project's directory. In 'test-output' folder you will find index.html. Open it in eclipse default browser and have a look.

TestNG has a very good documentation on each annotations and making the suite.xml file and for everything they support. Please follow here for further readings - TestNG Documentation

There is also a good documentation on Groups, Dependency Testing and Parameterization at above link provided. 

Open Source Web Application to Learn Test Automation – Sugar CRM:

Before starting to learn Selenium Webdriver, I strongly recommend you to download any of the Open Source web applications.  I suggest to use Sugar CRM here as this is a free edition.  We can just download the FastStack Windows Installer for MySQL and start using it.

You can download Sugar CRM from the following location: Sugar CRM

Download the latest version and install.

Installation of SugarCRM FastStack:

1. Download the SugarCRM FastStack from above given url.
2. Double Click the installer
3. Click next and just provide credentials for MySQL and initial login into SugarCRM and follow the install wizard.

Access the application using the URL – http://127.0.0.1/

Selenium Webdriver and TestNG setup in Eclipse:

This tutorial assumes that you are familiar with Java and you have some basic knowledge in coding in Java. Assumes that you are familiar with installations(Running setup files). In order to start setup we should get below listed tools which helps you to write and customize your automation tests.

Pre-requisites:

1. Eclipse - This is Java IDE where you will be writing your automation scripts. Download Eclipse IDE for Java EE Developers based on system configuration(32-bit or 64-bit).
2. Selenium Webdriver Java Client Libraries - These libraries are used to customize your automation tests. At the time of writing this post the Webdriver version is 2.39. You can update these jars as per the current versions available. I am also including most useful Jars along with Webdriver Jars. Download them from here.
3. TestNG - This is a test framework similar to JUnit. We will not use JUnit anywhere in our tutorials. I will explain how to integrate TestNG with eclipse later. TestNg plugin url for eclipse is,  

             For Eclipse 3.4 and above, enter http://beust.com/eclipse.
        For Eclipse 3.3 and below, enter http://beust.com/eclipse1.                        

Step by step guide to install TestNG in Eclipse:

1. Download Eclipse from the link provided and install
2. Open Eclipse from your installed location
3. Open the Install New Software from Help menu on Eclipse. It opens this dialog,
   
4. When above dialog appears click on Add button.
5. Write 'TestNg' in name field or you can name it to what ever you want and get TestNG appropriate plugin url provided above and insert in to Location field by clearing the default text on it. If your url is correct you will see TestNg installation files on the install new software dialog.
6. Select all check boxes and proceed with installation.
7. Choose to trust the software site and Restart Eclipse after completion.

Now you have TestNg installed on your Eclipse. Let's see how to setup Webdriver project in Eclipse.

Step by step guide to setup Webdriver project in Eclipse:

1. In Eclipse go to File -> New -> and select 'Java Project' if visible if not select 'Project'. You will see this dialog, 
   
2. Enter 'java' without quotes in Wizards input box and select 'Java Project' click Next.
3. Give any name to the project that we are going to create. Let's say selenium.demo and click finish. You will see selenium.demo project in the Package Explorer on Eclipse. Expand it.
4. Right click on src and select New -> Package.
5. Give it any name. Let's say com.selenium.webdriver. Click here to know best practice in naming the package.
6. Now right click on your package created in above step and select New -> Class.
7. Give it any name. Let's say WebdriverDemo. Usually the best practice is to use Upper Camel Case like WebdriverDemo. Try to use nouns because a class is normally representing something in the real world.
8. Leave everything default click Finish. See here, 
   
9. Now it's time to download the Selenium Webdriver libraries(Jars). Download it from above given link. Create directory called 'selenium' in C:\ drive. Move downloaded Jars to that directory. So path to the Jars is now 'C:\selenium'
10. Now right click on the project title and go to Build Path -> Configure Build Path. You will see this dialog, 
    
11. Select Libraries tab and click Add External JARs. This will open file explorer.
12. Go to the directory where Jars are located and select all of them and click open. Yours Jars are associated with your project now.
13. Double click on the class file created in step 7 in order to open it.
14. Copy the below code and paste it in your class.
15. Right click on the editor and select Run As -> TestNG Test.
16. If everything is fine, above code will launch the firefox browser, loads Google search page and prints 'Web page title: Google' in the console.

That's it, you are done with your setup. Follow my forthcoming posts to learn advanced usage of Webdriver.

If you really want to learn Selenium it's better to install an open source web application on your machine because they are stable. Here is a good application to automate - Sugar CRM.

See my post to know how to install Sugar CRM application and launch it - here.

Initializing the WebDriver object as a Thread Local for Parallel Test Execution:

When you have decided to run your selenium’s tests in parallel, your Webdriver object should be thread-safe i.e. a single object can be used with multiple threads at the same time without causing problems. To achieve thread-safe have your choices are to pass that object around to every method that needs (or may need) it, or to associate the object with the thread.

You may not want to fill up your method signatures with an additional parameter. In a non-threaded world, you could solve the problem with the Java equivalent of a global variable. In a threaded word, the equivalent of a global variable is a thread-local variable. 

In a single-threaded environment, Webdriver object will be a static field and return it as is. However, this will certainly not work in a multiple-threaded environment. Imagine if multiple threads used Webdriver object. Object used by each thread could overwrite each other since there is only one static instance of Webdriver. 

In order to solve this problem, ThreadLocal provides a very good solution,

Now, you have your Thread local Webdriver object. Let's see how can you use it in your tests.

Just call getDriver() method from above class in the blocks or methods where ever you want. This will invoke your Webdriver and launches the browser. usage,

You have to initialize your Webdriver object local to the method or block only. Do not initialize it as a field or global object. If you want to use driver object in your test it should be,

After finishing your test you have to remove or quit the driver object and close the browser session. You need to be very careful about cleaning up any ThreadLocal's you get()by using the ThreadLocal's remove() method. Like, you might want to remove or quit your ThreadLocal driver in teardown method like this,

That's it. You are done. You can call getDriver() any number of times and any where, it will return you the same object all the time and the scope is limited to that thread only.

Getting started with Selenium Webdriver:

In this post we will learn,

1. What is Selenium?
2. What is Selenium WebDriver?
3. Why/Where to use Webdriver?
4. Difference between Selenium IDE, RC and Webdriver?
5. Flexibility and Extensible of Webdriver?

What is Selenium?

Selenium is a browser based open source test automation tool which automates the user interactions with the browser. It is a suite of various tools. Selenium's tool suite includes,

• Selenium IDE:  Selenium IDE (Integrated Development Environment) is a prototyping tool for building test scripts. It is a Firefox plugin and provides an easy-to-use interface for developing automated tests. Selenium IDE has a recording feature, which records user actions as they are performed and then exports them as a reusable script in one of many programming languages that can be later executed. The IDE looks like this, 
  
                             
• Selenium RC: Selenium RC was the main Selenium project for a long time. Selenium works using javascript injection. If you have worked with selenium for sometime, then you must be aware of the same origin policy issues and limitations of javascript injection. Webdriver overcomes this by using a driver for each browser. For firefox this means, webdriver attaches itself to browser as an addon, for IE it uses automation atoms and for chrome and opera it uses the chrome driver. It ‘injects’ javascript functions into the browser when the browser was loaded and then used its javascript to drive the AUT within the browser.
• Selenium Webdriver:  WebDriver is designed to provide a simpler, more concise programming interface in addition to addressing some limitations in the Selenium-RC API. Selenium-WebDriver was developed to better support dynamic web pages where elements of a page may change without the page itself being reloaded. WebDriver’s goal is to supply a well-designed object-oriented API that provides improved support for modern advanced web-app testing problems.

What is Selenium WebDriver?

WebDriver is designed to provide a simpler, more concise programming interface in addition to addressing some limitations in the Selenium-RC API. Selenium-WebDriver was developed to better support dynamic web pages where elements of a page may change without the page itself being reloaded. WebDriver’s goal is to supply a well-designed object-oriented API that provides improved support for modern advanced web-app testing problems.

Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. How these direct calls are made, and the features they support depends on the browser you are using, i.e,

• Firefox:  Webdriver controls the Firefox browser using a Firefox plugin. The Firefox Profile that is used is stripped down from what is installed on the machine to only include the Selenium WebDriver.xpi (plugin).
• IE: This browser is controlled by a .dll and is thus only available on Windows OS. 
• Chrome:  This is maintained / supported by the Chromium project iteslf. WebDriver works with Chrome through the chromedriver binary (found on the chromium project’s download page). You need to have both chromedriver and a version of chrome browser installed. chromedriver needs to be placed somewhere on your system’s path in order for WebDriver to automatically discover it. The Chrome browser itself is discovered by chromedriver in the default installation path. These both can be overridden by environment variables. Please refer to the wiki for more information.

Why/Where to use Webdriver?

Why:

• Webdriver is a open-source tool.
• Provides various language APIs(i.e, Java, C#, Ruby..etc)
• Supports all major browsers( i.e, Firefox, Chrome, Internet Explorer and Opera and so on).
• Supports all major platforms( i.e, Windows, Linux, Mac and so on..)
• Can add user extensions.

Where:

Selenium Webdriver can be used to automate only web based applications but not desktop applications

Difference between Selenium IDE, RC and Webdriver?

Flexibility and Extensible of Webdriver?

Flexibility:
Selenium is highly flexible,

• There are many ways you can add functionality to both selenium test scripts and Selenium's framework to customize your test automation using own client libraries.
• In addition, since Selenium is open-source, the source code can be always downloaded and modified.
• Selenium supports for multiple programming and scripting languages allows the test automation engineer to build any logic they need in to their automated testing