Why does Gradle mark my build with failed tests as successful?

Question

I have a Gradle build with failing tests, but Gradle does not register the build as failed. However, the build does correctly fail if I remove a specific test case. Why is that?

$ ./gradlew clean check

:clean
:compileJava
:processResources UP-TO-DATE
:classes
:compileTestJava
:processTestResources UP-TO-DATE
:testClasses
:test

com.example.FooProgramCliTest > fail FAILED
    java.lang.AssertionError at FooProgramCliTest.java:26

1 test completed, 1 failed
:check

BUILD SUCCESSFUL

Code sample on GitHub, because it's a bit too bulky to meaningfully reproduce in the question body.

Answer 1

Make sure your tests do not inadvertently cause System.exit(int) to be called. Doing so will prevent Gradle from registering the test failure. See here for an example use case where this could happen.