Wednesday 5 March 2014

Learn TestNG:

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use. Junit and TestNG differ in the core design. Junit is a unit testing framework while TestNG addresses testing at a higher level. 

Let's discuss the three main differences between TestNG and Junit. 





  1. Dependency Test: In Junit, one test case failure can cause a bunch of test cases to fail in the test suite. There is no option of skipping the set of dependent test cases. The dependent test cases are also reported as failures. For example, suppose there is a test case to test login and the next 10 test cases need to perform a transaction after login. If the login test case fails the other 10 test cases will also fail. TestNG handles dependency between test cases. If one test case failure causes the failure of a group of test cases it skips that group and executes the rest of the test suite. The group that has dependency on the failed test cases is reported as skipped NOT failed.
  2. Groups: In TestNG groups can be defined. Groups are specific subsets of the test suite. We can choose to run only specific subset of the test-suite say database related test cases instead of running the entire test suite. In Junit for a long time it was not possible to run a specific subset of the test cases. We can either run the entire suite or run each test case individually. Junit 4.8 introduced a new feature called “Categories” to overcome this limitation. However groups are much easier to configure in TestNG.
  3. Parameterization: TestNG supports parameterization for objects. For example I can execute a single test case for multiple test data sets through the parameterization of DataProvider object. This makes the implementation of data driven testing more flexible in TestNG.

Annotations were formally added to the Java language in JDK 5 and TestNG made the choice to use annotations to annotate test classes.

Here is the list of annotations that TestNG supports:



AnnotationDescription
@BeforeSuiteThe annotated method will be run only once before all tests in this suite have run.
@AfterSuiteThe annotated method will be run only once after all tests in this suite have run.
@BeforeClassThe annotated method will be run only once before the first test method in the current class is invoked.
@AfterClassThe annotated method will be run only once after all the test methods in the current class have been run.
@BeforeTestThe annotated method will be run before any test method belonging to the classes inside the <test> tag is run.
@AfterTestThe annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.
@BeforeGroupsThe list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.
@AfterGroupsThe list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.
@BeforeMethodThe annotated method will be run before each test method.
@AfterMethodThe annotated method will be run after each test method.
@DataProviderMarks a method as supplying data for a test method. The annotated method must return an Object[ ][ ] where each Object[ ] can be assigned the parameter list of the test method. The @Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.
@FactoryMarks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return Object[ ].
@ListenersDefines listeners on a test class.
@ParametersDescribes how to pass parameters to a @Test method.
@TestMarks a class or a method as part of the test.

Let's see how we can use above annotations in out automated tests.

In order to continue further one should setup the TestNG plugin and add the TestNG jar file to the project build path in eclipse. If you don't know how to setup please follow the tutorial here - Selenium Webdriver and TestNG setup in Eclipse. You don't need to setup Selenium Webdriver at this stage, please follow instructions only related to TestNG.

Please follow the steps below to continue learning TestNG annotations.

  • Create a new project and a class to it by following Step by step guide to setup Webdriver project in Eclipse  in the above post. But change the project, package and class names accordingly.
  • Copy and paste the content below in to your new class.
  • Right click on the class file and run it as TestNG Test and look at the Eclipse Java console. This looks like below,
  • You have 3 tests in the class or in the test suite. @BeforeMethod and @AfterMethod have run 3 times before and after of every test case. These methods are called as configuration methods in TestNG. Generally in Selenium automation testing they are used for opening the browser before test and closing the browser after the test. And also you can you use it for opening and closing the resources.
  • @BeforeClass and @AfterClass are run only once for all the tests in that class. 
  • @BeforeSuite and @AfterSuite are run only once because they only run once per suite. A suite can have any no. of tests and classes, but these methods only runs per suite.
  • So, all of these annotations can be used accordingly as per your needs in your test suite.
  • At the beginning of the console logging you would have noticed a path to the xml file(C:\Users\username\AppData\Local\Temp\testng-eclipse-1346797963\testng-customsuite.xml). TestNG allows you to run your tests in several ways. TestNG xml is a XML file that describes the run-time definition of a test suite. It describes complex test definition while still remain easy to edit. 
  • Copy the xml file from the path given in console and paste in your project root directory. This should look like below screenshot, 
  • Open the xml file in eclipse and check how your test project is described in xml file.
  • Now right click on xml file and run as TestNG Test. You will get the same results as before.
  • TestNG also provides the default html report called 'test-output' for the test execution. This can be found in your project's directory. In 'test-output' folder you will find index.html. Open it in eclipse default browser and have a look.
TestNG has a very good documentation on each annotations and making the suite.xml file and for everything they support. Please follow here for further readings - TestNG Documentation

There is also a good documentation on Groups, Dependency Testing and Parameterization at above link provided. 





2 comments: