TestNG Listeners

TestNG Listeners are used to inspect and modify the testing behavior. TestNG listeners always extend org.testng.ITestNGListener marker interface. TestNG listeners can be defined for a test class using org.testng.annotations.Listeners annotation.

TestNG Listeners

There are many listeners interfaces provided by TestNG. They all extend org.testng.ITestNGListener interface.

Let’s look at some of the important TestNG listeners.

1. ISuiteListener: We can use this test suite listener to perform some operations when test suite starts and when all the tests are executed. This interface contains two methods – onStart(ISuite suite) and onFinish(ISuite suite) and provides access to test suite object.
2. ITestListener: We can use this listener to analyze test methods, perform logging. We can also use them to send notifications if any test fails by implementing onTestFailure(ITestResult result) method.
3. IAnnotationTransformer: We can implement this interface to modify the annotations for any @Test method. Note that we can use this annotation only with TestNG XML configuration.
4. IAnnotationTransformer2: We can implement this interface to modify the annotations for any method other than @Test method.This annotation can be used with TestNG XML configuration only.
5. IConfigurable: If a test class implements this interface, its run() method will be invoked instead of each configuration method found.
6. IConfigurationListener: Listener interface for events related to configuration methods.
7. IExecutionListener: This listener is used to monitor when a TestNG run starts and ends.
8. IHookable: If a test class implements this interface, its run() method will be invoked instead of each @Test method found.
9. IInvokedMethodListener: A listener that gets invoked before and after a method is invoked by TestNG.
10. IMethodInterceptor: This class is used to alter the list of test methods that TestNG is about to run.
11. IReporter: This interface can be implemented by clients to generate a report.

TestNG Listener Example

Let’s create a simple TestNG test class, then we will implement few listeners and add it to the class.

package com.journaldev.listeners;

import org.testng.annotations.DataProvider;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

public class Test3 {

	@Test
	public void test() {
		System.out.println("Test3 test method");
	}
	
	@Test(dataProvider = "dp")
	public void testString(String s) {
		System.out.println("Test3 testString method, input = "+s);
	}
	
	@DataProvider
	public Object[] dp() {
		return new Object[] {"A", "B"};
	}
	
}

When we run above as TestNG test, we get the following output in the console.

[RemoteTestNG] detected TestNG version 6.14.3
Test3 test method
Test3 testString method, input = A
Test3 testString method, input = B
PASSED: test
PASSED: testString("A")
PASSED: testString("B")

===============================================
    Default test
    Tests run: 3, Failures: 0, Skips: 0
===============================================


===============================================
Default suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================

ISuiteListener Example

Here is a simple implementation of ISuiteListener interface.

package com.journaldev.listeners;

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class Test3SuiteListener implements ISuiteListener {

	@Override
	public void onStart(ISuite suite) {
		System.out.println("TestNG suite default output directory = "+suite.getOutputDirectory());
		}

	@Override
	public void onFinish(ISuite suite) {
		System.out.println("TestNG invoked methods = " +suite.getAllInvokedMethods());
	}

}

ITestListener Example

Here is a simple implementation of ITestListener interface.

package com.journaldev.listeners;

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

public class Test3TestListener implements ITestListener{

	public Test3TestListener() {
		System.out.println("Test3Listener constructor");
	}

	@Override
	public void onTestStart(ITestResult result) {
		System.out.println("Test Started. "+result.getStartMillis());
	}

	@Override
	public void onTestSuccess(ITestResult result) {
		System.out.println("Test Success. "+result.getEndMillis());
	}

	@Override
	public void onTestFailure(ITestResult result) {
		System.out.println("Test Failed. "+result.getTestName());
	}

	@Override
	public void onTestSkipped(ITestResult result) {
		System.out.println("Test Skipped. "+result.getTestName());
	}

	@Override
	public void onTestFailedButWithinSuccessPercentage(ITestResult result) {		
	}

	@Override
	public void onStart(ITestContext context) {
		System.out.println("Context Name = "+context.getName());
	}

	@Override
	public void onFinish(ITestContext context) {
		System.out.println(context.getPassedTests());
	}
}

Now modify the Test3 class by adding Listeners to its class definition.

@Listeners({Test3TestListener.class, Test3SuiteListener.class})
public class Test3 {
}

When we run the class again as TestNG test, we get following output.

[RemoteTestNG] detected TestNG version 6.14.3
Test3Listener constructor
TestNG suite default output directory = /Users/pankaj/Documents/eclipse-github/TestNG-Examples/test-output/Default suite
Context Name = Default test
Test Started. 1527662805150
Test3 test method
Test Success. 1527662805159
Test Started. 1527662805163
Test3 testString method, input = A
Test Success. 1527662805164
Test Started. 1527662805165
Test3 testString method, input = B
Test Success. 1527662805166
[ResultMap map={[TestResult name=test status=SUCCESS method=Test3.test()[pri:0, instance:com.journaldev.listeners.Test3@291caca8] output={null}]=Test3.test()[pri:0, instance:com.journaldev.listeners.Test3@291caca8], [TestResult name=testString status=SUCCESS method=Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8] output={null}]=Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8], [TestResult name=testString status=SUCCESS method=Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8] output={null}]=Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8]}]
PASSED: test
PASSED: testString("A")
PASSED: testString("B")

===============================================
    Default test
    Tests run: 3, Failures: 0, Skips: 0
===============================================

TestNG invoked methods = [Test3.test()[pri:0, instance:com.journaldev.listeners.Test3@291caca8] 689745064, Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8]A  689745064, Test3.testString(java.lang.String)[pri:0, instance:com.journaldev.listeners.Test3@291caca8]B  689745064]

===============================================
Default suite
Total tests run: 3, Failures: 0, Skips: 0
===============================================

Summary

TestNG listeners are very powerful in monitoring the test suite and test cases. Some of them can be used to alter the testing behavior at runtime.