TestNG Tutorial

Filed Under: TestNG

TestNG is a testing framework for Java application that is inspired from JUnit and NUnit.

TestNG

TestNG provides many features that makes it easy to write and run test cases:

  • Annotation based
  • Plugins for major IDEs such as Eclipse and IDEA.
  • Automatically creation of test suites
  • HTML based report generation
  • Ability to run single test method using IDE plugins.
  • Data driven testing using @DataProvider
  • No additional dependencies to run and logging
  • TestNG XML file to run test suites with different configurations

TestNG Maven Dependency

TestNG libraries are present on maven repository, you can add below maven dependency to include TestNG in your project.


<dependency>
	<groupId>org.testng</groupId>
	<artifactId>testng</artifactId>
	<version>6.14.3</version>
	<scope>test</scope>
</dependency>

TestNG Eclipse Plugin

I am using Eclipse for TestNG example project, so let’s first install TestNG Eclipse plugin.

Go to “Eclipse Marketplace” and search for “TestNG”.

eclipse install testng plugin

Click on “Install” and go through further screens to install it. You will have to accept its license agreement and restart Eclipse once installation is finished.

Go to “Window | Show View | Other” and inside “Java” you should see “TestNG”.

eclipse testng view

This means that TestNG for Eclipse is successfully installed and we are ready to create our TestNG test classes.

Create TestNG Class

Go to your project in the explorer and click on “New | Other” and select “TestNG Class” and click Next.

eclipse new testng class

In next screen, we have to specify our TestNG class configurations and select annotations to include in the class and click on Finish button.

New TestNG Class package and annotations

This will generate TestNG skeleton class like below.


package com.journaldev.utils;

import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.AfterSuite;

public class TestUtils {
  @Test(dataProvider = "dp")
  public void f(Integer n, String s) {
  }
  @BeforeMethod
  public void beforeMethod() {
  }

  @AfterMethod
  public void afterMethod() {
  }


  @DataProvider
  public Object[][] dp() {
    return new Object[][] {
      new Object[] { 1, "a" },
      new Object[] { 2, "b" },
    };
  }
  @BeforeClass
  public void beforeClass() {
  }

  @AfterClass
  public void afterClass() {
  }

  @BeforeTest
  public void beforeTest() {
  }

  @AfterTest
  public void afterTest() {
  }

  @BeforeSuite
  public void beforeSuite() {
  }

  @AfterSuite
  public void afterSuite() {
  }

}

That’s it, we can run our basic TestNG class by going to “Run As | TestNG Test” as shown in below image.

Eclipse Run As TestNG Test

Since our test class has nothing, it should pass all tests. You can check “Console” for the logs generated and “TestNG View” will show you basic details of your test run.

TestNG Test Suite Flow

I have added some console logging in the annotated methods to understand the flow of TestNG test run.


package com.journaldev.utils;

import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.AfterSuite;

public class TestUtils {
  @Test(dataProvider = "dp")
  public void f(Integer n, String s) {
	  System.out.println("Inside @Test");
  }
  @BeforeMethod
  public void beforeMethod() {
	  System.out.println("Inside @BeforeMethod");
  }

  @AfterMethod
  public void afterMethod() {
	  System.out.println("Inside @AfterMethod");
  }


  @DataProvider
  public Object[][] dp() {
	  System.out.println("Inside @DataProvider");
    return new Object[][] {
      new Object[] { 1, "a" },
      new Object[] { 2, "b" },
    };
  }
  @BeforeClass
  public void beforeClass() {
	  System.out.println("Inside @BeforeClass");
  }

  @AfterClass
  public void afterClass() {
	  System.out.println("Inside @AfterClass");
  }

  @BeforeTest
  public void beforeTest() {
	  System.out.println("Inside @BeforeTest");
  }

  @AfterTest
  public void afterTest() {
	  System.out.println("Inside @AfterTest");
  }

  @BeforeSuite
  public void beforeSuite() {
	  System.out.println("Inside @BeforeSuite");
  }

  @AfterSuite
  public void afterSuite() {
	  System.out.println("Inside @AfterSuite");
  }

}

When I run it again, below console logs are generated.


[RemoteTestNG] detected TestNG version 6.14.3
Inside @BeforeSuite
Inside @BeforeTest
Inside @BeforeClass
Inside @DataProvider
Inside @BeforeMethod
Inside @Test
Inside @AfterMethod
Inside @BeforeMethod
Inside @Test
Inside @AfterMethod
Inside @AfterClass
Inside @AfterTest
PASSED: f(1, "a")
PASSED: f(2, "b")

===============================================
    Default test
    Tests run: 2, Failures: 0, Skips: 0
===============================================

Inside @AfterSuite

===============================================
Default suite
Total tests run: 2, Failures: 0, Skips: 0
===============================================

Based on above output, we can deduce the flow of TestNG test as:


BeforeSuite -> BeforeTest -> BeforeClass -> DataProvider -> 
{ BeforeMethod -> Test -> AfterMethod } * N 
-> AfterClass -> AfterTest -> AfterSuite

N is the number of times our test method is executed and it depends on the number of inputs generated by @DataProvider method.

We can use @BeforeXXX methods for some preprocessing and initialize resources that we want to use in the test suite. Similarly, we can use @AfterXXX methods for post-processing and closing resources.

TestNG Tutorial

Now that we have gone through the basics of TestNG and set it up in Eclipse, let’s create a class with few methods and use TestNG to run some test cases on it.


package com.journaldev.utils;

public class Utils {

	public static String NAME = "";

	public int add(int x, int y) {
		return x + y;
	}

	public int subtract(int x, int y) {
		return x - y;
	}

	public static void setName(String s) {
		System.out.println("Setting NAME to " + s);
		NAME = s;
	}
}

I have added few public methods, static and void methods too. Below is my TestNG class to test these methods.


package com.journaldev.utils;

import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class TestUtils {
	@Test(dataProvider = "dp")
	public void test_add(Integer x, Integer y) {
		Utils u = new Utils();
		Assert.assertEquals(u.add(x, y), x + y);
	}

	@Test(dataProvider = "dp")
	public void test_subtract(Integer x, Integer y) {
		Utils u = new Utils();
		Assert.assertEquals(u.subtract(x, y), x - y);
	}

	@Test(dataProvider = "dpName")
	public void test_setName(String s) {
		Utils.setName(s);
		Assert.assertEquals(Utils.NAME, s);
	}

	@DataProvider
	public Object[][] dp() {
		return new Object[][] { new Object[] { 1, 1 }, new Object[] { 2, 2 }, };
	}

	@DataProvider
	public Object[][] dpName() {
		return new Object[][] { new Object[] { "Utils" }, new Object[] { "MyUtils" }, };
	}

}

Now when we run above class as TestNG Test, we get following output in console.


[RemoteTestNG] detected TestNG version 6.14.3
Setting NAME to Utils
Setting NAME to MyUtils
PASSED: test_add(1, 1)
PASSED: test_add(2, 2)
PASSED: test_setName("Utils")
PASSED: test_setName("MyUtils")
PASSED: test_subtract(1, 1)
PASSED: test_subtract(2, 2)

===============================================
    Default test
    Tests run: 6, Failures: 0, Skips: 0
===============================================


===============================================
Default suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================

Refresh the project and you will see a newly generated directory test-output.

TestNG HTML Report

Open index.html in any browser and you should see HTML report like below image.

TestNG HTML Report Output

TestNG XML Suite

One of the best part of TestNG is the auto generation of test suite xml file.

TestNG XML test suite

We can edit it and add parameters to it, we can use it to run a single test method too. TestNG XML suite provides a lot of options and it’s very useful when you want to run a test case through the command line, such as on server with no Eclipse. We can run the TestNG test suite xml file using below command:


$java com.journaldev.utils.TestUtils testng.xml

TestNG XML is a very vast topic, if you want to explore more then go through its documentation.

TestNG Running Single Test Method

We can run a single test method too, this is very helpful when our test class has many methods and we are interested in only to test a few of them.

TestNG testing single method eclipse

TestNG Testing with Failures

Let’s see what happens when a bug is introduced in our code, change the Utils class setName method as below.


public static void setName(String s) {
	System.out.println("Setting NAME to " + s);
	// NAME = s; //introducing bug intentionally
}

Now when we run the test class again, we get following output in console.


[RemoteTestNG] detected TestNG version 6.14.3
Setting NAME to Utils
Setting NAME to MyUtils
FAILED: test_setName("Utils")
java.lang.AssertionError: expected [Utils] but found []
	at org.testng.Assert.fail(Assert.java:96)
	at org.testng.Assert.failNotEquals(Assert.java:776)
	at org.testng.Assert.assertEqualsImpl(Assert.java:137)
	at org.testng.Assert.assertEquals(Assert.java:118)
	at org.testng.Assert.assertEquals(Assert.java:453)
	at org.testng.Assert.assertEquals(Assert.java:463)
	at com.journaldev.utils.TestUtils.test_setName(TestUtils.java:23)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:114)
	at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)
	at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)

FAILED: test_setName("MyUtils")
java.lang.AssertionError: expected [MyUtils] but found []
	at org.testng.Assert.fail(Assert.java:96)
	at org.testng.Assert.failNotEquals(Assert.java:776)
	at org.testng.Assert.assertEqualsImpl(Assert.java:137)
	at org.testng.Assert.assertEquals(Assert.java:118)
	at org.testng.Assert.assertEquals(Assert.java:453)
	at org.testng.Assert.assertEquals(Assert.java:463)
	at com.journaldev.utils.TestUtils.test_setName(TestUtils.java:23)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:583)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
	at org.testng.TestRunner.privateRun(TestRunner.java:648)
	at org.testng.TestRunner.run(TestRunner.java:505)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:455)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415)
	at org.testng.SuiteRunner.run(SuiteRunner.java:364)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1137)
	at org.testng.TestNG.runSuites(TestNG.java:1049)
	at org.testng.TestNG.run(TestNG.java:1017)
	at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:114)
	at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:251)
	at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:77)

TestNG failure html report

Did you noticed that I am running test only on setName method.:)

Running TestNG Tests from Command Line

It’s good that we can run our test cases from Eclipse, but most of the times we want to run all the test cases of the project when building the project. Well, maven automatically runs the test cases for us. Below images show output of mvn clean install command.

Success Case
maven testng test cases run success

Failure Case
maven testng test cases run failure

TestNG Test Cases Order

TestNG test cases are run in the order of their method names. So in our case the order of execution would be: test_add -> test_setName -> test_subtract.

We can change the order by setting the priority of test methods. If we want the order of execution as test_add -> test_subtract -> test_setName then we can change our @Test annotation like below.


@Test(dataProvider = "dp", priority=1)
public void test_add(Integer x, Integer y) {
}

@Test(dataProvider = "dp", priority=2)
public void test_subtract(Integer x, Integer y) {
}

@Test(dataProvider = "dpName", priority=3)
public void test_setName(String s) {
}

Notice that lower priority methods are executed first.

Summary

TestNG is a very popular unit testing framework and I really liked that it’s very easy to setup and use in the project. We explored basics of TestNG and looked into most common examples to use it in our project.

You can download the TestNG tutorial project from our GitHub Repository.

Comments

  1. Mohammed Jinnah says:

    This is awesome! Most comprehensive TestNG tutorial I have come across.

  2. nakidde damalie says:

    thanks so much sir. really good and so helpfull

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages