Chapter 57. Multi-project Builds

http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects

The powerful support for multi-project builds is one of Gradle's unique selling points. This topic is also the most intellectually challenging.

A multi-project build in gradle consists of one root project, and one or more subprojects that may also have subprojects.

57.1. Cross project configuration

While each subproject could configure itself in complete isolation of the other subprojects, it is common that subprojects share common traits. It is then usually preferable to share configurations among projects, so the same configuration affects several subprojects.

Let's start with a very simple multi-project build. Gradle is a general purpose build tool at its core, so the projects don't have to be java projects. Our first examples are about marine life.

57.1.1. Configuration and execution

Section 56.1, “Build phases” describes the phases of every Gradle build. Let's zoom into the configuration and execution phases of a multi-project build. Configuration here means executing thebuild.gradle file of a project, which implies e.g. downloading all plugins that were declared using 'apply plugin'. By default, the configuration of all projects happens before any task is executed. This means that when a single task, from a single project is requested, all projects of multi-project build are configured first. The reason every project needs to be configured is to support the flexibility of accessing and changing any part of the Gradle project model.

57.1.1.1. Configuration on demand

The Configuration injection feature and access to the complete project model are possible because every project is configured before the execution phase. Yet, this approach may not be the most efficient in a very large multi-project build. There are Gradle builds with a hierarchy of hundreds of subprojects. The configuration time of huge multi-project builds may become noticeable. Scalability is an important requirement for Gradle. Hence, starting from version 1.4 a new incubating 'configuration on demand' mode is introduced.

Configuration on demand mode attempts to configure only projects that are relevant for requested tasks, i.e. it only executes the build.gradle file of projects that are participating in the build. This way, the configuration time of a large multi-project build can be reduced. In the long term, this mode will become the default mode, possibly the only mode for Gradle build execution. The configuration on demand feature is incubating so not every build is guaranteed to work correctly. The feature should work very well for multi-project builds that have decoupled projects (Section 57.9, “Decoupled Projects”). In “configuration on demand” mode, projects are configured as follows:

• The root project is always configured. This way the typical common configuration is supported (allprojects or subprojects script blocks).
• The project in the directory where the build is executed is also configured, but only when Gradle is executed without any tasks. This way the default tasks behave correctly when projects are configured on demand.
• The standard project dependencies are supported and makes relevant projects configured. If project A has a compile dependency on project B then building A causes configuration of both projects.
• The task dependencies declared via task path are supported and cause relevant projects to be configured. Example: someTask.dependsOn(":someOtherProject:someOtherTask")
• A task requested via task path from the command line (or Tooling API) causes the relevant project to be configured. For example, building 'projectA:projectB:someTask' causes configuration of projectB.

Eager to try out this new feature? To configure on demand with every build run see Section 20.1, “Configuring the build environment via gradle.properties”. To configure on demand just for a given build please see Appendix D, Gradle Command Line.

57.1.2. Defining common behavior

Let's look at some examples with the following project tree. This is a multi-project build with a root project named water and a subproject named bluewhale.

Example 57.1. Multi-project tree - water & bluewhale projects

Build layout

water/
  build.gradle
  settings.gradle
  bluewhale/

Note: The code for this example can be found at samples/userguide/multiproject/firstExample/water which is in both the binary and source distributions of Gradle.

settings.gradle

include 'bluewhale'

And where is the build script for the bluewhale project? In Gradle build scripts are optional. Obviously for a single project build, a project without a build script doesn't make much sense. For multiproject builds the situation is different. Let's look at the build script for the water project and execute it:

Example 57.2. Build script of water (parent) project

build.gradle

Closure cl = { task -> println "I'm $task.project.name" }
task hello << cl
project(':bluewhale') {
    task hello << cl
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale

Gradle allows you to access any project of the multi-project build from any build script. The Project API provides a method called project(), which takes a path as an argument and returns the Project object for this path. The capability to configure a project build from any build script we call cross project configuration. Gradle implements this via configuration injection.

We are not that happy with the build script of the water project. It is inconvenient to add the task explicitly for every project. We can do better. Let's first add another project called krill to our multi-project build.

Example 57.3. Multi-project tree - water, bluewhale & krill projects

Build layout

water/
  build.gradle
  settings.gradle
  bluewhale/
  krill/

Note: The code for this example can be found at samples/userguide/multiproject/addKrill/water which is in both the binary and source distributions of Gradle.

settings.gradle

include 'bluewhale', 'krill'

Now we rewrite the water build script and boil it down to a single line.

Example 57.4. Water project build script

build.gradle

allprojects {
    task hello << { task -> println "I'm $task.project.name" }
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
I'm krill

Is this cool or is this cool? And how does this work? The Project API provides a property allprojects which returns a list with the current project and all its subprojects underneath it. If you callallprojects with a closure, the statements of the closure are delegated to the projects associated with allprojects. You could also do an iteration via allprojects.each, but that would be more verbose.

Other build systems use inheritance as the primary means for defining common behavior. We also offer inheritance for projects as you will see later. But Gradle uses configuration injection as the usual way of defining common behavior. We think it provides a very powerful and flexible way of configuring multiproject builds.

Another possibilty for sharing configuration is to use a common external script. See Section 14.3, “Configuring the project using an external build script” for more information.

57.2. Subproject configuration

The Project API also provides a property for accessing the subprojects only.

57.2.1. Defining common behavior

Example 57.5. Defining common behavior of all projects and subprojects

build.gradle

allprojects {
    task hello << {task -> println "I'm $task.project.name" }
}
subprojects {
    hello << {println "- I depend on water"}
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
- I depend on water
I'm krill
- I depend on water

You may notice that there are two code snippets referencing the “hello” task. The first one, which uses the “task” keyword, constructs the task and provides it's base configuration. The second piece doesn't use the “task” keyword, as it is further configuring the existing “hello” task. You may only construct a task once in a project, but you may any number of code blocks providing additional configuration.

57.2.2. Adding specific behavior

You can add specific behavior on top of the common behavior. Usually we put the project specific behavior in the build script of the project where we want to apply this specific behavior. But as we have already seen, we don't have to do it this way. We could add project specific behavior for the bluewhale project like this:

Example 57.6. Defining specific behaviour for particular project

build.gradle

allprojects {
    task hello << {task -> println "I'm $task.project.name" }
}
subprojects {
    hello << {println "- I depend on water"}
}
project(':bluewhale').hello << {
    println "- I'm the largest animal that has ever lived on this planet."
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
- I depend on water
- I'm the largest animal that has ever lived on this planet.
I'm krill
- I depend on water

As we have said, we usually prefer to put project specific behavior into the build script of this project. Let's refactor and also add some project specific behavior to the krill project.

Example 57.7. Defining specific behaviour for project krill

Build layout

water/
  build.gradle
  settings.gradle
  bluewhale/
    build.gradle
  krill/
    build.gradle

Note: The code for this example can be found at samples/userguide/multiproject/spreadSpecifics/water which is in both the binary and source distributions of Gradle.

settings.gradle

include 'bluewhale', 'krill'

bluewhale/build.gradle

hello.doLast {
  println "- I'm the largest animal that has ever lived on this planet."
}

krill/build.gradle

hello.doLast {
  println "- The weight of my species in summer is twice as heavy as all human beings."
}

build.gradle

allprojects {
    task hello << {task -> println "I'm $task.project.name" }
}
subprojects {
    hello << {println "- I depend on water"}
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
- I depend on water
- I'm the largest animal that has ever lived on this planet.
I'm krill
- I depend on water
- The weight of my species in summer is twice as heavy as all human beings.

57.2.3. Project filtering

To show more of the power of configuration injection, let's add another project called tropicalFish and add more behavior to the build via the build script of the water project.

57.2.3.1. Filtering by name

Example 57.8. Adding custom behaviour to some projects (filtered by project name)

Build layout

water/
  build.gradle
  settings.gradle
  bluewhale/
    build.gradle
  krill/
    build.gradle
  tropicalFish/

Note: The code for this example can be found at samples/userguide/multiproject/addTropical/water which is in both the binary and source distributions of Gradle.

settings.gradle

include 'bluewhale', 'krill', 'tropicalFish'

build.gradle

allprojects {
    task hello << {task -> println "I'm $task.project.name" }
}
subprojects {
    hello << {println "- I depend on water"}
}
configure(subprojects.findAll {it.name != 'tropicalFish'}) {
    hello << {println '- I love to spend time in the arctic waters.'}
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
- I depend on water
- I love to spend time in the arctic waters.
- I'm the largest animal that has ever lived on this planet.
I'm krill
- I depend on water
- I love to spend time in the arctic waters.
- The weight of my species in summer is twice as heavy as all human beings.
I'm tropicalFish
- I depend on water

The configure() method takes a list as an argument and applies the configuration to the projects in this list.

57.2.3.2. Filtering by properties

Using the project name for filtering is one option. Using extra project properties is another. (See Section 13.4.2, “Extra properties” for more information on extra properties.)

Example 57.9. Adding custom behaviour to some projects (filtered by project properties)

Build layout

water/
  build.gradle
  settings.gradle
  bluewhale/
    build.gradle
  krill/
    build.gradle
  tropicalFish/
    build.gradle

Note: The code for this example can be found at samples/userguide/multiproject/tropicalWithProperties/water which is in both the binary and source distributions of Gradle.

settings.gradle

include 'bluewhale', 'krill', 'tropicalFish'

bluewhale/build.gradle

ext.arctic = true
hello.doLast {
  println "- I'm the largest animal that has ever lived on this planet."
}

krill/build.gradle

ext.arctic = true
hello.doLast {
    println "- The weight of my species in summer is twice as heavy as all human beings."
}

tropicalFish/build.gradle

ext.arctic = false

build.gradle

allprojects {
    task hello << {task -> println "I'm $task.project.name" }
}
subprojects {
    hello {
        doLast {println "- I depend on water"}
        afterEvaluate { Project project ->
            if (project.arctic) { doLast {
                println '- I love to spend time in the arctic waters.' }
            }
        }
    }
}

Output of gradle -q hello

> gradle -q hello
I'm water
I'm bluewhale
- I depend on water
- I'm the largest animal that has ever lived on this planet.
- I love to spend time in the arctic waters.
I'm krill
- I depend on water
- The weight of my species in summer is twice as heavy as all human beings.
- I love to spend time in the arctic waters.
I'm tropicalFish
- I depend on water

In the build file of the water project we use an afterEvaluate notification. This means that the closure we are passing gets evaluated after the build scripts of the subproject are evaluated. As the property arctic is set in those build scripts, we have to do it this way. You will find more on this topic in Section 57.6, “Dependencies - Which dependencies?”

57.3. Execution rules for multi-project builds

When we executed the hello task from the root project dir, things behaved in an intuitive way. All the hello tasks of the different projects were executed. Let's switch to the bluewhale dir and see what happens if we execute Gradle from there.

Example 57.10. Running build from subproject

Output of gradle -q hello

> gradle -q hello
I'm bluewhale
- I depend on water
- I'm the largest animal that has ever lived on this planet.
- I love to spend time in the arctic waters.

The basic rule behind Gradle's behavior is simple. Gradle looks down the hierarchy, starting with the current dir, for tasks with the name hello and executes them. One thing is very important to note. Gradle always evaluates every project of the multi-project build and creates all existing task objects. Then, according to the task name arguments and the current dir, Gradle filters the tasks which should be executed. Because of Gradle's cross project configuration every project has to be evaluated before any task gets executed. We will have a closer look at this in the next section. Let's now have our last marine example. Let's add a task to bluewhale and krill.

Example 57.11. Evaluation and execution of projects

bluewhale/build.gradle

ext.arctic = true
hello << { println "- I'm the largest animal that has ever lived on this planet." }

task distanceToIceberg << {
    println '20 nautical miles'
}

krill/build.gradle

ext.arctic = true
hello << {
    println "- The weight of my species in summer is twice as heavy as all human beings."
}

task distanceToIceberg << {
    println '5 nautical miles'
}

Output of gradle -q distanceToIceberg

> gradle -q distanceToIceberg
20 nautical miles
5 nautical miles

Here's the output without the -q option:

Example 57.12. Evaluation and execution of projects

Output of gradle distanceToIceberg

> gradle distanceToIceberg
:bluewhale:distanceToIceberg
20 nautical miles
:krill:distanceToIceberg
5 nautical miles

BUILD SUCCESSFUL

Total time: 1 secs

The build is executed from the water project. Neither water nor tropicalFish have a task with the name distanceToIceberg. Gradle does not care. The simple rule mentioned already above is: Execute all tasks down the hierarchy which have this name. Only complain if there is no such task!

57.4. Running tasks by their absolute path

As we have seen, you can run a multi-project build by entering any subproject dir and execute the build from there. All matching task names of the project hierarchy starting with the current dir are executed. But Gradle also offers to execute tasks by their absolute path (see also Section 57.5, “Project and task paths”):

Example 57.13. Running tasks by their absolute path

Output of gradle -q :hello :krill:hello hello

> gradle -q :hello :krill:hello hello
I'm water
I'm krill
- I depend on water
- The weight of my species in summer is twice as heavy as all human beings.
- I love to spend time in the arctic waters.
I'm tropicalFish
- I depend on water

The build is executed from the tropicalFish project. We execute the hello tasks of the water, the krill and the tropicalFish project. The first two tasks are specified by their absolute path, the last task is executed using the name matching mechanism described above.

57.5. Project and task paths

A project path has the following pattern: It starts with an optional colon, which denotes the root project. The root project is the only project in a path that is not specified by its name. The rest of a project path is a colon-separated sequence of project names, where the next project is a subproject of the previous project.

The path of a task is simply its project path plus the task name, like “:bluewhale:hello”. Within a project you can address a task of the same project just by its name. This is interpreted as a relative path.

Originally Gradle used the '/' character as a natural path separator. With the introduction of directory tasks (see Section 14.1, “Directory creation”) this was no longer possible, as the name of the directory task contains the '/' character.

57.6. Dependencies - Which dependencies?

The examples from the last section were special, as the projects had no Execution Dependencies. They had only Configuration Dependencies. The following sections illustrate the differences between these two types of dependencies.

57.6.1. Execution dependencies

57.6.1.1. Dependencies and execution order

Example 57.14. Dependencies and execution order

Build layout

messages/
  settings.gradle
  consumer/
    build.gradle
  producer/
    build.gradle

Note: The code for this example can be found at samples/userguide/multiproject/dependencies/firstMessages/messages which is in both the binary and source distributions of Gradle.

settings.gradle

include 'consumer', 'producer'

consumer/build.gradle

task action << {
    println("Consuming message: ${rootProject.producerMessage}")
}

producer/build.gradle

task action << {
    println "Producing message:"
    rootProject.producerMessage = 'Watch the order of execution.'
}

Output of gradle -q action

> gradle -q action
Consuming message: null
Producing message:

This didn't quite do what we want. If nothing else is defined, Gradle executes the task in alphanumeric order. Therefore, Gradle will execute “:consumer:action” before “:producer:action”. Let's try to solve this with a hack and rename the producer project to “aProducer”.

Example 57.15. Dependencies and execution order

Build layout

messages/
  settings.gradle
  aProducer/
    build.gradle
  consumer/
    build.gradle

settings.gradle

include 'consumer', 'aProducer'

aProducer/build.gradle

task action << {
    println "Producing message:"
    rootProject.producerMessage = 'Watch the order of execution.'
}

consumer/build.gradle

task action << {
    println("Consuming message: ${rootProject.producerMessage}")
}

Output of gradle -q action

> gradle -q action
Producing message:
Consuming message: Watch the order of execution.

We can show where this hack doesn't work if we now switch to the consumer dir and execute the build.

Example 57.16. Dependencies and execution order

Output of gradle -q action

> gradle -q action
Consuming message: null

The problem is that the two “action” tasks are unrelated. If you execute the build from the “messages” project Gradle executes them both because they have the same name and they are down the hierarchy. In the last example only one “action” task was down the hierarchy and therefore it was the only task that was executed. We need something better than this hack.

57.6.1.2. Declaring dependencies

Example 57.17. Declaring dependencies

Build layout

messages/
  settings.gradle
  consumer/
    build.gradle
  producer/
    build.gradle

Note: The code for this example can be found at samples/userguide/multiproject/dependencies/messagesWithDependencies/messages which is in both the binary and source distributions of Gradle.

settings.gradle

include 'consumer', 'producer'

consumer/build.gradle

task action(dependsOn: ":producer:action") << {
    println("Consuming message: ${rootProject.producerMessage}")
}

producer/build.gradle

task action << {
    println "Producing message:"
    rootProject.producerMessage = 'Watch the order of execution.'
}

Output of gradle -q action

> gradle -q action
Producing message:
Consuming message: Watch the order of execution.

Running this from the consumer directory gives:

Example 57.18. Declaring dependencies

Output of gradle -q action

> gradle -q action
Producing message:
Consuming message: Watch the order of execution.

This is now working better because we have declared that the “action” task in the “consumer” project has an execution dependency on the “action” task in the “producer” project.

57.6.1.3. The nature of cross project task dependencies

Of course, task dependencies across different projects are not limited to tasks with the same name. Let's change the naming of our tasks and execute the build.

Example 57.19. Cross project task dependencies

consumer/build.gradle

task consume(dependsOn: ':producer:produce') << {
    println("Consuming message: ${rootProject.producerMessage}")
}

producer/build.gradle

task produce << {
    println "Producing message:"
    rootProject.producerMessage = 'Watch the order of execution.'
}

Output of gradle -q consume

> gradle -q consume
Producing message:
Consuming message: Watch the order of execution.

57.6.2. Configuration time dependencies

Let's see one more example with our producer-consumer build before we enter Java land. We add a property to the “producer” project and create a configuration time dependency from “consumer” to “producer”.

Example 57.20. Configuration time dependencies

consumer/build.gradle

def message = rootProject.producerMessage

task consume << {
    println("Consuming message: " + message)
}

producer/build.gradle

rootProject.producerMessage = 'Watch the order of evaluation.'

Output of gradle -q consume

> gradle -q consume
Consuming message: null

The default evaluation order of projects is alphanumeric (for the same nesting level). Therefore the “consumer” project is evaluated before the “producer” project and the “producerMessage” value is set after it is read by the “consumer” project. Gradle offers a solution for this.

Example 57.21. Configuration time dependencies - evaluationDependsOn

consumer/build.gradle

evaluationDependsOn(':producer')

def message = rootProject.producerMessage

task consume << {
    println("Consuming message: " + message)
}

Output of gradle -q consume

> gradle -q consume
Consuming message: Watch the order of evaluation.

The use of the “evaluationDependsOn” command results in the evaluation of the “producer” project before the “consumer” project is evaluated. This example is a bit contrived to show the mechanism. In this case there would be an easier solution by reading the key property at execution time.

Example 57.22. Configuration time dependencies

consumer/build.gradle

task consume << {
    println("Consuming message: ${rootProject.producerMessage}")
}

Output of gradle -q consume

> gradle -q consume
Consuming message: Watch the order of evaluation.

Configuration dependencies are very different from execution dependencies. Configuration dependencies are between projects whereas execution dependencies are always resolved to task dependencies. Also note that all projects are always configured, even when you start the build from a subproject. The default configuration order is top down, which is usually what is needed.

To change the default configuration order to “bottom up”, use the “evaluationDependsOnChildren()” method instead.

On the same nesting level the configuration order depends on the alphanumeric position. The most common use case is to have multi-project builds that share a common lifecycle (e.g. all projects use the Java plugin). If you declare with dependsOn a execution dependency between different projects, the default behavior of this method is to also create a configuration dependency between the two projects. Therefore it is likely that you don't have to define configuration dependencies explicitly.

57.6.3. Real life examples

Gradle's multi-project features are driven by real life use cases. One good example consists of two web application projects and a parent project that creates a distribution including the two web applications. ^[21] For the example we use only one build script and do cross project configuration.

Example 57.23. Dependencies - real life example - crossproject configuration

Build layout

webDist/
  settings.gradle
  build.gradle
  date/
    src/main/java/
      org/gradle/sample/
        DateServlet.java
  hello/
    src/main/java/
      org/gradle/sample/
        HelloServlet.java

Note: The code for this example can be found at samples/userguide/multiproject/dependencies/webDist which is in both the binary and source distributions of Gradle.

settings.gradle

include 'date', 'hello'

build.gradle

allprojects {
    apply plugin: 'java'
    group = 'org.gradle.sample'
    version = '1.0'
}

subprojects {
    apply plugin: 'war'
    repositories {
        mavenCentral()
    }
    dependencies {
        compile "javax.servlet:servlet-api:2.5"
    }
}

task explodedDist(dependsOn: assemble) << {
    File explodedDist = mkdir("$buildDir/explodedDist")
    subprojects.each {project ->
        project.tasks.withType(Jar).each {archiveTask ->
            copy {
                from archiveTask.archivePath
                into explodedDist
            }
        }
    }
}

We have an interesting set of dependencies. Obviously the date and hello projects have a configuration dependency on webDist, as all the build logic for the webapp projects is injected by webDist. The execution dependency is in the other direction, as webDist depends on the build artifacts of date and hello. There is even a third dependency. webDist has a configuration dependency on dateand hello because it needs to know the archivePath. But it asks for this information at execution time. Therefore we have no circular dependency.

Such dependency patterns are daily bread in the problem space of multi-project builds. If a build system does not support these patterns, you either can't solve your problem or you need to do ugly hacks which are hard to maintain and massively impair your productivity as a build master.

57.7. Project lib dependencies

What if one project needs the jar produced by another project in its compile path, and not just the jar but also the transitive dependencies of this jar? Obviously this is a very common use case for Java multi-project builds. As already mentioned in Section 51.4.3, “Project dependencies”, Gradle offers project lib dependencies for this.

Example 57.24. Project lib dependencies

Build layout

java/
  settings.gradle
  build.gradle
  api/
    src/main/java/
      org/gradle/sample/
        api/
          Person.java
        apiImpl/
          PersonImpl.java
  services/personService/
    src/
      main/java/
        org/gradle/sample/services/
          PersonService.java
      test/java/
        org/gradle/sample/services/
          PersonServiceTest.java
  shared/
    src/main/java/
      org/gradle/sample/shared/
        Helper.java

Note: The code for this example can be found at samples/userguide/multiproject/dependencies/java which is in both the binary and source distributions of Gradle.

We have the projects “shared”, “api” and “personService”. The “personService” project has a lib dependency on the other two projects. The “api” project has a lib dependency on the “shared” project. ^[22]

Example 57.25. Project lib dependencies

settings.gradle

include 'api', 'shared', 'services:personService'

build.gradle

subprojects {
    apply plugin: 'java'
    group = 'org.gradle.sample'
    version = '1.0'
    repositories {
        mavenCentral()
    }
    dependencies {
        testCompile "junit:junit:4.11"
    }
}

project(':api') {
    dependencies {
        compile project(':shared')
    }
}

project(':services:personService') {
    dependencies {
        compile project(':shared'), project(':api')
    }
}

All the build logic is in the “build.gradle” file of the root project. ^[23] A “lib” dependency is a special form of an execution dependency. It causes the other project to be built first and adds the jar with the classes of the other project to the classpath. It also adds the dependencies of the other project to the classpath. So you can enter the “api” directory and trigger a “gradle compile”. First the “shared” project is built and then the “api” project is built. Project dependencies enable partial multi-project builds.

If you come from Maven land you might be perfectly happy with this. If you come from Ivy land, you might expect some more fine grained control. Gradle offers this to you:

Example 57.26. Fine grained control over dependencies

build.gradle

subprojects {
    apply plugin: 'java'
    group = 'org.gradle.sample'
    version = '1.0'
}

project(':api') {
    configurations {
        spi
    }
    dependencies {
        compile project(':shared')
    }
    task spiJar(type: Jar) {
        baseName = 'api-spi'
        dependsOn classes
        from sourceSets.main.output
        include('org/gradle/sample/api/**')
    }
    artifacts {
        spi spiJar
    }
}

project(':services:personService') {
    dependencies {
        compile project(':shared')
        compile project(path: ':api', configuration: 'spi')
        testCompile "junit:junit:4.11", project(':api')
    }
}

The Java plugin adds per default a jar to your project libraries which contains all the classes. In this example we create an additional library containing only the interfaces of the “api” project. We assign this library to a new dependency configuration. For the person service we declare that the project should be compiled only against the “api” interfaces but tested with all classes from “api”.

57.7.1. Disabling the build of dependency projects

Sometimes you don't want depended on projects to be built when doing a partial build. To disable the build of the depended on projects you can run Gradle with the -a option.

57.8. Parallel project execution

With more and more CPU cores available on developer desktops and CI servers, it is important that Gradle is able to fully utilise these processing resources. More specifically, the parallel execution attempts to:

• Reduce total build time for a multi-project build where execution is IO bound or otherwise does not consume all available CPU resources.
• Provide faster feedback for execution of small projects without awaiting completion of other projects.

Although Gradle already offers parallel test execution via Test.setMaxParallelForks() the feature described in this section is parallel execution at a project level. Parallel execution is an incubating feature. Please use it and let us know how it works for you.

Parallel project execution allows the separate projects in a decoupled multi-project build to be executed in parallel (see also: Section 57.9, “Decoupled Projects”). While parallel execution does not strictly require decoupling at configuration time, the long-term goal is to provide a powerful set of features that will be available for fully decoupled projects. Such features include:

• Section 57.1.1.1, “Configuration on demand”.
• Configuration of projects in parallel.
• Re-use of configuration for unchanged projects.
• Project-level up-to-date checks.
• Using pre-built artifacts in the place of building dependent projects.

How does parallel execution work? First, you need to tell Gradle to use the parallel mode. You can use the command line argument (Appendix D, Gradle Command Line) or configure your build environment (Section 20.1, “Configuring the build environment via gradle.properties”). Unless you provide a specific number of parallel threads Gradle attempts to choose the right number based on available CPU cores. Every parallel worker exclusively owns a given project while executing a task. This means that 2 tasks from the same project are never executed in parallel. Therefore only multi-project builds can take advantage of parallel execution. Task dependencies are fully supported and parallel workers will start executing upstream tasks first. Bear in mind that the alphabetical scheduling of decoupled tasks, known from the sequential execution, does not really work in parallel mode. You need to make sure the task dependencies are declared correctly to avoid ordering issues.

57.9. Decoupled Projects

Gradle allows any project to access any other project during both the configuration and execution phases. While this provides a great deal of power and flexibility to the build author, it also limits the flexibility that Gradle has when building those projects. For instance, this effectively prevents Gradle from correctly building multiple projects in parallel, configuring only a subset of projects, or from substituting a pre-built artifact in place of a project dependency.

Two projects are said to be decoupled if they do not directly access each other's project model. Decoupled projects may only interact in terms of declared dependencies: project dependencies (Section 51.4.3, “Project dependencies”) and/or task dependencies (Section 6.5, “Task dependencies”). Any other form of project interaction (i.e. by modifying another project object or by reading a value from another project object) causes the projects to be coupled. The consequence of coupling during the configuration phase is that if gradle is invoked with the 'configuration on demand' option, the result of the build can be flawed in several ways. The consequence of coupling during execution phase is that if gradle is invoked with the parallel option, one project task runs too late to influence a task of a project building in parallel. Gradle does not attempt to detect coupling and warn the user, as there are too many possibilities to introduce coupling.

A very common way for projects to be coupled is by using configuration injection (Section 57.1, “Cross project configuration”). It may not be immediately apparent, but using key Gradle features like the allprojects and subprojects keywords automatically cause your projects to be coupled. This is because these keywords are used in a build.gradle file, which defines a project. Often this is a “root project” that does nothing more than define common configuration, but as far as Gradle is concerned this root project is still a fully-fledged project, and by using allprojects that project is effectively coupled to all other projects. Coupling of the root project to subprojects does not impact 'configuration on demand', but using the allprojects and subprojects in any subproject'sbuild.gradle file will have an impact.

This means that using any form of shared build script logic or configuration injection (allprojects, subprojects, etc.) will cause your projects to be coupled. As we extend the concept of project decoupling and provide features that take advantage of decoupled projects, we will also introduce new features to help you to solve common use cases (like configuration injection) without causing your projects to be coupled.

In order to make good use of cross project configuration without running into issues for parallel and 'configuration on demand' options, follow these recommendations:

• Avoid a subproject's build.gradle referencing other subprojects; prefering cross configuration from the root project.
• Avoid changing the configuration of other projects at execution time.

57.10. Multi-Project Building and Testing

The build task of the Java plugin is typically used to compile, test, and perform code style checks (if the CodeQuality plugin is used) of a single project. In multi-project builds you may often want to do all of these tasks across a range of projects. The buildNeeded and buildDependents tasks can help with this.

Look at Example 57.25, “Project lib dependencies”. In this example, the “:services:personservice” project depends on both the “:api” and “:shared” projects. The “:api” project also depends on the “:shared” project.

Assume you are working on a single project, the “:api” project. You have been making changes, but have not built the entire project since performing a clean. You want to build any necessary supporting jars, but only perform code quality and unit tests on the project you have changed. The build task does this.

Example 57.27. Build and Test Single Project

Output of gradle :api:build

> gradle :api:build
:shared:compileJava
:shared:processResources
:shared:classes
:shared:jar
:api:compileJava
:api:processResources
:api:classes
:api:jar
:api:assemble
:api:compileTestJava
:api:processTestResources
:api:testClasses
:api:test
:api:check
:api:build

BUILD SUCCESSFUL

Total time: 1 secs

While you are working in a typical development cycle repeatedly building and testing changes to the “:api” project (knowing that you are only changing files in this one project), you may not want to even suffer the expense of building “:shared:compile” to see what has changed in the “:shared” project. Adding the “-a” option will cause Gradle to use cached jars to resolve any project lib dependencies and not try to re-build the depended on projects.

Example 57.28. Partial Build and Test Single Project

Output of gradle -a :api:build

> gradle -a :api:build
:api:compileJava
:api:processResources
:api:classes
:api:jar
:api:assemble
:api:compileTestJava
:api:processTestResources
:api:testClasses
:api:test
:api:check
:api:build

BUILD SUCCESSFUL

Total time: 1 secs

If you have just gotten the latest version of source from your version control system which included changes in other projects that “:api” depends on, you might want to not only build all the projects you depend on, but test them as well. The buildNeeded task also tests all the projects from the project lib dependencies of the testRuntime configuration.

Example 57.29. Build and Test Depended On Projects

Output of gradle :api:buildNeeded

> gradle :api:buildNeeded
:shared:compileJava
:shared:processResources
:shared:classes
:shared:jar
:api:compileJava
:api:processResources
:api:classes
:api:jar
:api:assemble
:api:compileTestJava
:api:processTestResources
:api:testClasses
:api:test
:api:check
:api:build
:shared:assemble
:shared:compileTestJava
:shared:processTestResources
:shared:testClasses
:shared:test
:shared:check
:shared:build
:shared:buildNeeded
:api:buildNeeded

BUILD SUCCESSFUL

Total time: 1 secs

You also might want to refactor some part of the “:api” project that is used in other projects. If you make these types of changes, it is not sufficient to test just the “:api” project, you also need to test all projects that depend on the “:api” project. The buildDependents task also tests all the projects that have a project lib dependency (in the testRuntime configuration) on the specified project.

Example 57.30. Build and Test Dependent Projects

Output of gradle :api:buildDependents

> gradle :api:buildDependents
:shared:compileJava
:shared:processResources
:shared:classes
:shared:jar
:api:compileJava
:api:processResources
:api:classes
:api:jar
:api:assemble
:api:compileTestJava
:api:processTestResources
:api:testClasses
:api:test
:api:check
:api:build
:services:personService:compileJava
:services:personService:processResources
:services:personService:classes
:services:personService:jar
:services:personService:assemble
:services:personService:compileTestJava
:services:personService:processTestResources
:services:personService:testClasses
:services:personService:test
:services:personService:check
:services:personService:build
:services:personService:buildDependents
:api:buildDependents

BUILD SUCCESSFUL

Total time: 1 secs

Finally, you may want to build and test everything in all projects. Any task you run in the root project folder will cause that same named task to be run on all the children. So you can just run “gradle build” to build and test all projects.

57.11. Multi Project and buildSrc

Section 60.3, “Build sources in the buildSrc project” tells us that we can place build logic to be compiled and tested in the special buildSrc directory. In a multi project build, there can only be onebuildSrc directory which must be located in the root directory.

57.12. Property and method inheritance

Properties and methods declared in a project are inherited to all its subprojects. This is an alternative to configuration injection. But we think that the model of inheritance does not reflect the problem space of multi-project builds very well. In a future edition of this user guide we might write more about this.

Method inheritance might be interesting to use as Gradle's Configuration Injection does not support methods yet (but will in a future release).

You might be wondering why we have implemented a feature we obviously don't like that much. One reason is that it is offered by other tools and we want to have the check mark in a feature comparison :). And we like to offer our users a choice.

57.13. Summary

Writing this chapter was pretty exhausting and reading it might have a similar effect. Our final message for this chapter is that multi-project builds with Gradle are usually not difficult. There are five elements you need to remember: allprojects, subprojects, evaluationDependsOn, evaluationDependsOnChildren and project lib dependencies. ^[24] With those elements, and keeping in mind that Gradle has a distinct configuration and execution phase, you already have a lot of flexibility. But when you enter steep territory Gradle does not become an obstacle and usually accompanies and carries you to the top of the mountain.

---

[21] The real use case we had, was using http://lucene.apache.org/solr, where you need a separate war for each index you are accessing. That was one reason why we have created a distribution of webapps. The Resin servlet container allows us, to let such a distribution point to a base installation of the servlet container.

[22] “services” is also a project, but we use it just as a container. It has no build script and gets nothing injected by another build script.

[23] We do this here, as it makes the layout a bit easier. We usually put the project specific stuff into the build script of the respective projects.

[24] So we are well in the range of the 7 plus 2 Rule:)

Previous|Contents|Next