9 Easy TestNG Features You Can Learn Today That Will Improve Your Test Automation

The TestNG unit testing framework has many easy-to-implement features that will make you test automation easier:

1. Dependent scripts
2. Soft assertions
3. Skipped tests
4. Execute tests multiple times
5. Before/After Method, Class, Test, Group, Suite
6. Tests with parameters
7. Run tests in parallel
8. Reports
9. Run failed tests

Most of these features do not exist in JUNIT.

INSTALL TEST NG

First, lets see how Test NG can be installed:

• In Eclipse, open Eclipse Marketplace (Help menu --> Eclipse MarketPlace)

• Select the Popular tab in the Eclipse MarketPlace window

• Find the Test NG for Eclipse Plugin

• Install it

• Add Test NG to the project:

• Right click on the project name and select Properties

• In the Properties page, select the Java Build Path tab, select Libraries and click the Add Library button

• Select Test NG and click Finish

It is a good practice to have only independent test scripts when working on test automation.

In some cases, however, it is useful to have the ability of making test scripts dependent of other scripts.

For example, let's assume that we have a number of test scripts that all depend on a database being available.

If the database is not available, all test scripts fail.

To avoid having all test scripts failing, we could create a test script that checks the database status.

Then, we can make all other test scripts depend on the "check database status" script.

The result will be that, if the database is not available, the other test scripts will not fail but be ignored.


It is not possible to have dependent scripts in JUNIT.


Test NG allows creating dependent tests using methods and groups.

TEST SCRIPTS THAT DEPEND ON METHODS

package tests;

import static org.testng.AssertJUnit.assertTrue;
import org.testng.annotations.Test;

public class OrderedCheckOutTests {

@Test (dependsOnMethods = "addProductToCart")
public void checkOutAndPay()
{

System.out.println("3. checkOutAndPay Completed ");  

}

@Test (dependsOnMethods = "searchForProduct")
public void addProductToCart()
{

System.out.println("2. addProductToCart Completed");

}

@Test
public void searchForProduct()
{

System.out.println("1. searchForProduct Completed");

assertTrue(false);

}

}

When executing the test class, the test scripts run in the following order:

1. searchForProduct()
2. addProductToCart() (depends on searchForProduct so searchForProduct runs first)
3. checkOutAndPay() (depends on addProductToCart so addProductToCart runs first)

TEST SCRIPTS THAT DEPEND ON GROUPS

package tests;

import org.testng.annotations.Test;

public class OrderedCheckOutTests2 {

@Test (dependsOnGroups = "search")
public void checkOutAndPay()
{

System.out.println("3. checkOutAndPay Completed ");  

}

@Test (groups = "search")
public void addProductToCart()
{

System.out.println("2. addProductToCart Completed");

}

@Test (groups = "search")
public void searchForProduct()
{

System.out.println("1. searchForProduct Completed");  

}

}

When executing the test class, the test scripts run in the following order:

1. searchForProduct()
2. addProductToCart() (the first 2 scripts can run in any order)
3. checkOutAndPay() (depends on searchForProduct and addProductToCart so these 2 scripts run first)

An example will help explaining how soft assertions work.

We are working on automating testing for a user registration page.

There are multiple fields in the page for username, password, first name, last name, email, etc.

The test script checks if each field value is valid so it has an assertion for each field.

If multiple fields have invalid values, the assertion for the first invalid field fails and the remaining assertions are ignored.

It would be nice if we would have the ability to run assertions for all fields and then 

• fail the script if at least one assertion fails

• pass the script if all assertions pass

This is what soft assertions do:

package tests;

import static org.testng.AssertJUnit.assertTrue;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class SoftAssertions {

SoftAssert softAssertion = new SoftAssert();

@Test
public void testHardAssert1()
{

assertTrue(false);

} 

@Test
public void testHardAssert2()
{

assertTrue(false);

assertTrue(false);

assertTrue(true);

}

@Test 
public void testSoftAssert2()
{

softAssertion.assertTrue(false);

softAssertion.assertEquals(1, 2);

softAssertion.assertEquals(true, false);

softAssertion.assertAll();

}

} 

When running the test class, we get the following results:

1. testHardAssert1() fails
2. testHardAssert2() fails (only the first assertion is executed)
3. testSoftAssert2() fails (all 3 assertions are executed)

Soft assertions work as containers for normal assertions.

If any of the child assertions fails, the soft assertion fails after all child assertions run.

If all child assertions pass, the soft assertion passes as well.

Lets go back to the example where multiple test scripts depend on a database being available.

Lets assume that each test script checks first if the database is available.

If the database is not available, it would be nice to flag the tests as skipped instead of failed.

The SkipException from Test NG can be used for skipping tests:

package tests;

import org.testng.SkipException;
import org.testng.annotations.Test;

public class SkippedTests {

@Test
public void checkOutAndPay()
{

System.out.println("3. checkOutAndPay Completed ");

}

@Test
public void addProductToCart()
{

System.out.println("2. addProductToCart Completed");

throw new SkipException("Skipping this exception");

}

@Test
public void searchForProduct()
{

System.out.println("1. searchForProduct Completed"); 

if (1 != 2)
throw new SkipException("Skipping this exception");

}

}

If a test script needs to be executed multiple times, the @Test (invocationCount = 10) annotation can be used as shown in the example below.

package tests;

import org.testng.annotations.Test;

@Test (groups = "regression")
public class TestsExecutedMultipleTimes {

@Test (invocationCount = 10)
public void test1()
{

}

@Test
public void test2()
{

}

}

In Test NG, the following components are being used:

• Method

• Suite: made of one or more tests

• Test: made of one or more classes

• Class: made of one or more methods

• Group: made of one or more methods of different classes

Test NG has Before and After annotations for all these components (compared with JUNIT which has them only for methods, classes and suites):

• @BeforeSuite / @AfterSuite — before a suite starts / after all the test methods in a certain suite have been run

• @BeforeTest / @AfterTest — before a test starts / after all the test methods in a certain test have been run (remember that a test is made of

• @BeforeClass / @AfterClass — before a test class starts / after all the test methods in a certain class have been run

• @BeforeMethod / @AfterMethod — before a test method is run / after a test method has been run

• @BeforeGroups / @AfterGroups — before any test method in a given group is run / after all the test methods in a given group have been run

Tests with parameters exist in JUNIT as well.

Test NG adds more functionality for them.

The parameter values can come from

• the Test NG XML file

• the test script uses  @Parameters ({ "parameter1", "parameter2"}) for specifying the parameter names

• multiple data providers; this is an improvement compared to JUNIT where only 1 provider is possible

• the test script uses @Test(dataProvider = "provider1") for specifying the data provider that it needs

• the test script can have parameters like any other method (not possible in JUNIT)

TEST NG XML FILE




<suite name="Suite1" verbose="1" >
<parameter name="parameter1" value="value 1" />
<parameter name="parameter2" value="value 2" />
</suite>






package tests;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Parameters;
import org.testng.annotations.Test;

public class TestsWithParameters {

@DataProvider(name = "provider1")
public Object[][] provider1() {

return new Object[][] {
{ "provider1_aaa", "provider1_bbb" },
{ "provider1_ccc", "provider1_ddd" },
{ "provider1_eee", "provider1_fff" },
{ "provider1_ggg", "provider1_hhh"},
{ "provider1_iii", "provider1_jjj" },

};

}

@DataProvider(name = "provider2")
public Object[][] provider2() {

return new Object[][] {
{ "provider2_aaa", "provider2_bbb" },
{ "provider2_ccc", "provider2_ddd" },
{ "provider2_eee", "provider2_fff" },
{ "provider2_ggg", "provider2_hhh"},
{ "provider2_iii", "provider2_jjj" },

};

}

@Parameters ({ "parameter1", "parameter2"})
@Test
public void displayParameterValuesFromFile(String p1, String p2) {

System.out.println(p1 + " , " + p2);

}

@Test(dataProvider = "provider1")
public void displayParameterValuesFromDataProvider1(String p1, String p2) {

System.out.println(p1 + " , " + p2);

}

@Test(dataProvider = "provider2")
public void displayParameterValuesFromDataProvider2(String p1, String p2) {

System.out.println(p1 + " , " + p2);

}

}

When time is an issue and the execution of the test suite is slow, running test scripts in parallel will help.

Running test scripts in parallel in Test NG is simpler than doing it in JUNIT.

And it does not require using Maven or Maven SureFire.

You just need to specify values for the invocation count and the threadPoolSize parameters.

package tests;

import static org.testng.AssertJUnit.assertTrue;
import org.testng.annotations.Test;

public class ParallelTests {

@Test(invocationCount = 5, threadPoolSize = 10)
public void test1()
{

assertTrue (true);

}

@Test(invocationCount = 5, threadPoolSize = 10)
public void test2()
{

assertTrue (true);

}

@Test(invocationCount = 5, threadPoolSize = 10)
public void test3()
{

assertTrue (true);

}

@Test(invocationCount = 5, threadPoolSize = 10)
public void test4()
{

assertTrue (true);

}

@Test(invocationCount = 5, threadPoolSize = 10)
public void test5()
{

assertTrue (true);

}

}

Test NG allows running the failed test scripts without making any changes to the test annotations.

This is so much easier than in JUNIT where you need to ignore the passing tests to be able to run the failed ones.

An example will make this clear.

Lets start with a simple class with 6 test scripts.

import static org.testng.AssertJUnit.assertTrue;

import org.testng.Assert;
import org.testng.annotations.Test;
public class class1 {
 
@Test
public void test1()
{
Assert.assertTrue(true);
}

@Test
public void test2()
{
Assert.assertTrue(true);
}

@Test
public void test3()
{
Assert.assertTrue(true);
}

@Test
public void test4()
{
Assert.assertTrue(true);
}

@Test
public void test5()
{
Assert.assertTrue(true);
}

@Test
public void test6()
{
Assert.assertTrue(true);
}

}

Running the class makes all test scripts pass:

[TestNG] Running: C:\Users\home\AppData\Local\Temp\testng-eclipse--1476802653\testng-customsuite.xml

PASSED: test1
PASSED: test2
PASSED: test3
PASSED: test4
PASSED: test5
PASSED: test6

==============================================

Default test Tests run: 6, Failures: 0, Skips: 0 ==============================================

Default suite Total tests run: 6, Failures: 0, Skips: 0

==============================================



Lets make 2 of the test scripts fail (test2 and test4).

import static org.testng.AssertJUnit.assertTrue;

import org.testng.Assert;
import org.testng.annotations.Test;
public class class1 {
 
@Test
public void test1()
{
Assert.assertTrue(true);
}

@Test
public void test2()
{
Assert.assertTrue(false);
}

@Test
public void test3()
{
Assert.assertTrue(true);
}

@Test
public void test4()
{
Assert.assertTrue(false);
}

@Test
public void test5()
{
Assert.assertTrue(true);
}

@Test
public void test6()
{
Assert.assertTrue(true);
}

}

Re-running the class shows 2 test scripts failing:

=============================================== 

 Default test Tests run: 6, Failures: 2, Skips: 0 

=============================================== 

Default suite Total tests run: 6, Failures: 2, Skips: 0 

===============================================

To re-run only the failed scripts, we need to

• fix the errors from the failed tests

• select the Results Of Running Suite Tab

• click the Run Failed Test Button

The failed tests only are being re-executed.

JUNIT does not have any default reporting functionality.

Reports can be generated in different ways

• use a 3rd party framework 

• create them from scratch

Test NG has a simple report built in.

To see it, follow the next steps:

• click the Results Of Running Suite Tab

• click the Open Test NG Report button

The report is displayed in a new tab of the Eclipse editor.

Customized information can be added to the Reporter Output section using the Reporter.log(text) method:

Share this post