How to use JaCoCo Gradle Plugin with TestNG and JUnit?

Question

I am working on a project where we are using TestNG and JUnit for our tests.

Unfortunately when writing TestNG tests they are not considered in JaCoCo Coverage Reports.

I wrote a testng.gradle file, which I include in each of the build.gradle files (it's a multi-module project):

task testNG(type: Test) { useTestNG() }
test.dependsOn testNG

Both JUnit and TestNG tests work this way.

If I write my testng.gradle like this:

test {
    useTestNG()
}

JaCoCo works properly, but obviously only TestNG tests get executed.

How can I fix it? Is it a bug in Gradle's JaCoCo plugin?

Answer 1

Seems that while Gradle JaCoCo Plugin enhances testNG task, so that its execution uses JaCoCo Java agent, but it forgets to update jacocoTestReport task, so that this task doesn't use results of execution of testNG task. Don't know if this is a bug or on purpose, but solution is provided below.

to demonstrate this

file src/main/java/Example.java:

public class Example {
  public void junit() {
    System.out.println("JUnit");
  }

  public void testng() {
    System.out.println("TestNG");
  }
}

file src/test/java/ExampleJUnitTest.java:

import org.junit.Test;

public class ExampleJUnitTest {
  @Test
  public void test() {
    new Example().junit();
  }
}

file src/test/java/ExampleTestNGTest.java:

import org.testng.annotations.Test;

public class ExampleTestNGTest {
  @Test
  public void test() {
    new Example().testng();
  }
}

file build.gradle:

apply plugin: 'java'
apply plugin: 'jacoco'

repositories {
  mavenCentral()
}

dependencies {
  testCompile 'org.testng:testng:6.8.8'
  testCompile 'junit:junit:4.12'
}

task testNG(type: Test) {
  useTestNG()
}

test {
  dependsOn testNG
}

After execution of gradle clean test jacocoTestReport -d you'll see in log

java ... -javaagent:.../jacocoagent.jar=destfile=build/jacoco/testNG.exec ...
...
java ... -javaagent:.../jacocoagent.jar=destfile=build/jacoco/test.exec ...

and that directory build/jacoco contains two files - testNG.exec and test.exec, for testNG and for test tasks respectively. While JaCoCo report shows only execution of JUnit by test task.

to solve this

Either instruct task testNG to write execution data into same file as test:

task testNG(type: Test) {
  useTestNG()
  jacoco {
    destinationFile = file("$buildDir/jacoco/test.exec")
  }
}

Either instruct task jacocoTestReport to also use testNG.exec file:

jacocoTestReport {
  executionData testNG
}

I'm assuming that the same should be done for the case of multi-module project in general and in your case in particular, since Minimal, Complete, and Verifiable example of your multi-module project setup wasn't provided.

Answer 2

The second solution from @Godin's fixed it for me. But to add on that, I found that testNG is not recognized by gradle since it is not matching the right *.exec file under $buildDir/jacoco. In my case it is testNg.exec, so it worked after I include this block in my subproject's build.gradle:

jacocoTestReport {
  executionData testNg
}

The first solution does not work for me

task testNG(type: Test) {
  useTestNG()
  jacoco {
    destinationFile = file("$buildDir/jacoco/test.exec")
  }
}

I guess it is because the testNG is overwriting the whole file $buildDir/jacoco/test.exec generated from JUnit.