Replacing deprecated `Project#exec` in `doFirst`/`doLast`

Question

I've got a custom Gradle task like this:

tasks.register("myTask") {
  doLast {
    exec { commandLine("sh", "-c", "myscript.sh") }
    exec { commandLine("sh", "-c", "myscript2.sh") }
  }
}

When I run it, I get this warning:

The Project.exec(Action) method has been deprecated. This is scheduled to be removed in Gradle 9.0. Use ExecOperations.exec(Action) or ProviderFactory.exec(Action) instead. Consult the upgrading guide for further information: https://docs.gradle.org/8.11.1/userguide/upgrading_version_8.html#deprecated_project_exec

I've read the documentation there, but it's quite confusing. What should I use to replace this?

Answer 1

tl;dr:

1. You need to decide whether you're executing your task at execution time or configuration time. (If you don't know, your task is probably running at execution time.)
2. If you're running at execution time, you'll either need to copy and paste some boilerplate into your build script, and then replace your exec calls with execOps.exec calls, or convert your task into a class.
3. If you're running at configuration time, you'll want to use providers.exec. But beware, its API is very different from the old exec you've used before.

The docs are confusing

https://docs.gradle.org/current/userguide/upgrading_version_8.html#deprecated_project_exec has some documentation, but it's hard to understand unless you already know a lot about Gradle.

The Project#exec(Closure), Project#exec(Action), Project#javaexec(Closure), Project#javaexec(Action) methods have been deprecated and will be removed in Gradle 9.0.

These methods are scheduled for removal as part of the ongoing effort to make writing configuration-cache-compatible code easier. There is no way to use these methods without breaking configuration cache requirements so it is recommended to migrate to a compatible alternative. The appropriate replacement for your use case depends on the context in which the method was previously called.

At execution time, for example in @TaskAction or doFirst/doLast callbacks, the use of Project instance is not allowed when the configuration cache is enabled. To run external processes, tasks should use an injected ExecOperation service, which has the same API and can act as a drop-in replacement. The standard Java/Groovy/Kotlin process APIs, like java.lang.ProcessBuilder can be used as well.

At configuration time, only special Provider-based APIs must be used to run external processes when the configuration cache is enabled. You can use ProviderFactory.exec and ProviderFactory.javaexec to obtain the output of the process. A custom ValueSource implementation can be used for more sophisticated scenarios. The configuration cache guide has a more elaborate example of using these APIs.

Execution time vs. configuration time

If you're like most Gradle users, you might never have had to wrestle with the distinction between execution time and configuration time.

The docs define it here.

https://docs.gradle.org/current/userguide/partr2_build_lifecycle.html

A Gradle build has three distinct phases:

Phase 1 - Initialization

During the initialization phase, Gradle determines which projects will take part in the build, and creates a Project instance for each project.

Phase 2 - Configuration

During the configuration phase, the Project objects are configured using the build scripts of all projects in the build. Gradle determines the set of tasks to be executed.

Phase 3 - Execution

During the execution phase, Gradle executes each of the selected tasks.

Gradle is gradually working on a "configuration cache." https://docs.gradle.org/current/userguide/configuration_cache.html

The configuration cache is a feature that significantly improves build performance by caching the result of the configuration phase and reusing this for subsequent builds. Using the configuration cache, Gradle can skip the configuration phase entirely when nothing that affects the build configuration, such as build scripts, has changed. Gradle also applies performance improvements to task execution as well.

Making the configuration phase cacheable requires splitting Project.exec into two different exec functions, one for configuration time, and one for execution time.

• If the result of your exec is designed to enable/disable tasks or dependencies, it needs to run at configuration time.
• Otherwise, if your exec runs after the task list has been defined (especially in a doLast block), then it runs at execution time.

Try turning on the configuration cache

You can run ./gradlew --configuration-cache to turn on the cache just once. To turn it on permanently, you can add org.gradle.configuration-cache=true to your gradle.properties.

If you can get your build to work at that point, you'll find that the build runs a lot faster.

But you'll have to fix this Project#exec issue to get it to work, so, let's get started.

Running processes at execution time

You've got (at least) three options for running processes at execution time.

Execution Time Option 1: Use ExecOperations in doFirst/doLast with some boilerplate

If your code looks like this:

// kotlin example
tasks.register("myTask") {
  doLast {
    exec { commandLine("sh", "-c", "myscript.sh") }
    exec { commandLine("sh", "-c", "myscript2.sh") }
  }
}

// groovy example
task myTask {
  doLast {
    exec { commandLine "sh" "-c" "myscript.sh" }
    exec { commandLine "sh" "-c" "myscript2.sh" }
  }
}

You can add some boilerplate at the top of your build file, and then call execOps.exec, like this:

// kotlin example
interface InjectedExecOps {
    @get:Inject val execOps: ExecOperations
}

// ...

tasks.register("myTask") {
  val injected = project.objects.newInstance<InjectedExecOps>()
  doLast {
    injected.execOps.exec { commandLine("sh", "-c", "myscript.sh") }
    injected.execOps.exec { commandLine("sh", "-c", "myscript2.sh") }
  }
}

// groovy example
interface InjectedExecOps {
    @Inject //@javax.inject.Inject
    ExecOperations getExecOps()
}

// ...

task myTask {
  def injected = project.objects.newInstance(InjectedExecOps)
  doLast {
    injected.execOps.exec { commandLine "sh" "-c" "myscript.sh" }
    injected.execOps.exec { commandLine "sh" "-c" "myscript2.sh" }
  }
}

This code may be quite unfamiliar to you as a Gradle user. If you're not neck-deep in modern Gradle code, you might never have used injected services in Gradle. Well, if you want to understand this code, you'll have to understand how services and dependency injection work in Gradle.

You can read the "injected services" link above for more details, but the idea is that to get access to an ExecOperations object, you have to declare a class or interface and then have Gradle inject ExecOperations into it. The injection has to be performed by instantiating the injectable object with ObjectFactory#newInstance. and Gradle provides an ObjectFactory as project.objects. Once the object is instantiated (and injected), you can extract the injected ExecOperations object and use it in much the same way you would have used project#exec.

Execution Time Option 2: If all you're doing is executing one process, maybe just run an Exec task directly

Gradle tasks are normally meant to be full-blown classes; you can do everything with "ad-hoc tasks," empty classes with doFirst / doLast callbacks, but Gradle wasn't really meant to work that way.

Gradle's Exec task is meant to be run like this:

// kotlin example
tasks.register<Exec>("myTask") {
  commandLine "sh" "-c" "myscript.sh"
}

// groovy example
task myTask(type: Exec) {
  commandLine "sh" "-c" "myscript.sh"
}

That'll work great as long as you're not using any custom logic, e.g. if (foo) { exec { ... } }. Exec tasks work just the same in Gradle 8 and Gradle 9.

Execution Time Option 3: Build a custom task class with @TaskAction and inject ExecOperations into it

// kotlin example
abstract class MyExecOperationsTask
@Inject constructor(private var execOperations: ExecOperations) : DefaultTask() {

    @TaskAction
    fun doTaskAction() {
        execOperations.exec {
            commandLine("sh", "-c", "myscript.sh")
        }
    }
}

tasks.register("myInjectedExecOperationsTask", MyExecOperationsTask::class) {}

// groovy example
abstract class MyExecOperationsTask extends DefaultTask {
    private ExecOperations execOperations

    @Inject //@javax.inject.Inject
    MyExecOperationsTask(ExecOperations execOperations) {
        this.execOperations = execOperations
    }

    @TaskAction
    void doTaskAction() {
        execOperations.exec {
            commandLine "sh" "-c" "myscript.sh"
        }
    }
}

tasks.register("myInjectedExecOperationsTask", MyExecOperationsTask) {}

This is the very most official way to define a Gradle task. It probably doesn't make sense to do it this way unless you're planning to distribute your task to third parties, e.g. in a Gradle plugin or something.

(Defining custom classes right there in a build script normally doesn't add any benefit over simpler approaches. It's just more code to accomplish the same thing.)

Configuration time: Use providers.exec

At configuration time, you want to use the configuration-time variant of exec, ProviderFactory#exec. Gradle provides a ProviderFactory as project.providers, or just providers for short.

Beware, the API of providers.exec is quite different from Project#exec.

1. providers.exec executes lazily. You have to call .get() on its output's result, standardOutput, or standardError, or it won't run at all.
2. providers.exec doesn't pipe standardOutput or standardError to the Gradle log. You have to manually read it and log it if you want to see what it says.
3. As a result, it can be difficult to figure out why a providers.exec execution fails.

Running a simple providers.exec call

providers.exec {
  commandLine("sh", "-c", "myscript.sh")
}.result.get()

Reading standard output (only)

// kotlin example
val gitVersion = providers.exec {
    commandLine("git", "--version")
}.standardOutput.asText.get()

// groovy example
def gitVersion = providers.exec {
    commandLine("git", "--version")
}.standardOutput.asText.get()

Reading exit code, standard output, and standard error

// kotlin example
val execOutput = providers.exec {
  commandLine("sh", "-c", "echo OK; echo OOPS >&2; exit 1")
  isIgnoreExitValue = true
}

val exitCode = execOutput.result.get().exitValue
val stdout = execOutput.standardOutput.asText.get()
val stderr = execOutput.standardError.asText.get()

println("STDOUT: $stdout")
println("STDERR: $stderr")
println("Exit Code: $exitCode")

if (exitCode != 0) {
  throw GradleException("Command failed with exit code $exitCode")
}

With a ValueSource class

Apparently you can get even fancier than this by defining a ValueSource class, but I've never used it, and I have no idea why you'd bother.

// kotlin example
abstract class GitVersionValueSource : ValueSource<String, ValueSourceParameters.None> {
    @get:Inject
    abstract val execOperations: ExecOperations

    override fun obtain(): String {
        val output = ByteArrayOutputStream()
        execOperations.exec {
            commandLine("git", "--version")
            standardOutput = output
        }
        return String(output.toByteArray(), Charset.defaultCharset())
    }
}

// groovy example
abstract class GitVersionValueSource implements ValueSource<String, ValueSourceParameters.None> {
    @Inject
    abstract ExecOperations getExecOperations()

    String obtain() {
        ByteArrayOutputStream output = new ByteArrayOutputStream()
        execOperations.exec {
            it.commandLine "git", "--version"
            it.standardOutput = output
        }
        return new String(output.toByteArray(), Charset.defaultCharset())
    }
}

I dunno. If you need this, hopefully you'll know that you need it.