https://gradle.com/ logo
Join Slack
Powered by
I have a multi app build, and I want to run a give...
# community-support
v
I have a multi app build, and I want to run a givem task over all of the apps. Which of these is the preferred way? A) to declare a task in root project, which explicistly lists them
Copy code
// Root
tasks.register("lintAllDebug") {
    group = "verification"
    dependsOn(
            ":foo:app:lintAllDebug",
            ":bar:app:lintAllDebug",
            ":quax:app:lintAllDebug"
    )
}
B) rely on the same named-ness and just 
./gradlew lintAllDebug
v
Manly depends on you I'd say. If it is only for usage from commandline or IDE integration that is. If you for example want to depend on it from another task in the root project, or maybe want to include that build into another composite build and depend on these tasks there, the root-level lifecycle task might make sense.
But if not needed otherwise, I'd probably prefer B
v
so if I need something to depend on it, it needs to be a explicit task, right? what if in that case, theres way to many "child" tasks to declare, isnt there some automatic way? or in other words, can I keep the no-setup of 
./gradlew lintAllDebug
way and let it be depended on?
v
You mean like 
tasks.check { dependsOn(somehowAllTasksNamedLintAllDebug) }
in the root project?
v
yes
Copy code
tasks.register("lintAllDebug") {
    group = "verification"
    dependsOn(... magic)
}
v
No, not really how you imagine it afaik. You could for example ensure each and every project has that task, even if it is a no-op and then depend on the task in all projects. Or you could have a separate task that is available in each and every project and then depends on that task in the relevant projects, depending on that extra task in all projects. Or you could iterate through 
allprojects
and for example depend on the task in all projects that are named 
app
if that is sufficiently identifying. ...
I mean, there is a way, you should just not use it. 😄
Especially in the light of asking about isolated projects before.
v
yes, because currently I have bunch of app level tasks that iterate through all child projects and get references to the tasks, and place that in the dependsOn and im trying to rework it to make it IP friendly
You could for example ensure each and every project has that task, even if it is a no-op and then depend on the task in all projects.
how would I do the depending? sounds kinda like what I'm already doing, if you mean some for loop
v
Yes
subprojects.forEach { dependsOn("${it.path}:lintAllDebug") }
As long as you do not access mutable information of the projects like trying to use 
tasks
on it or similar, it should be IP-safe. The project name and project path are immutable once the settings script is finished, so that should be IP-safe I think.
v
Copy code
// root
tasks.register("lintAllDebug") {
	subprojects.forEach { dependsOn("${it.path}:lintAllDebug") }
}
do you mean like this?
v
If each and every subproject has a task with that name, yes.
v
and if not, then to query that given subproject's tasks, is breaking IP, right? I mean to check if 
it.tasks
contains the task
v
It is a very bad idea even without IP. Cross-project access of mutable state is almost as bad as cross-project configuration. But yes, IP it will definitely break. That's why I said, if possible you could filter on the project name.
v
shame I cannot access the same mechanism the 
./gradlew foo
can in that case it will only pickup projects that have the task
v
Yep
Maybe there is a feature request for it, but I don't remember having seen one. Maybe you should create one if not.
v
okay ill take that in mind one more please - say every module declares 
test
task, and I want to run test task only on given subset of modules, but I dont want to keep a explicit list because it doesnt scale would it be smart to have just a "wrapper" task, just to declare the task via my custom name and then aggregate that?
Copy code
tasks.register("myTest") {
    dependsOn "test"
}
so I can then 
./gradlew myTest
? or is this "token" task not recommended
v
That's a lifecycle task, and I'd say that would be perfectly fine.
v
okay maybe not test then but something custom 
custom
and then 
myCustom
as to "annotate" the projects, so their tasks can then be pulled up by the natural aggregator, if you know what I mean
v
I don't see where the difference is to 
test
and 
myTest
Not 
test
is the lifecycle task, 
myTest
is the lifecycle task
v
a task that just declares dependsOn and nothing else, thats called a lifecycle task?
v
lifecycle task = task that does not do work but its purpose is to depend on other tasks, so that you can call those tasks by calling the lifecycle task
And lifecycle tasks are practically the only cases where an explicit 
dependsOn
is not a code-smell
In practically all other cases it is usually a sign that you did not properly wire task outputs to task inputs which comes with the necessary implicit task dependencies automatically.
v
intersting name, I'd not figure
Copy code
// Root
tasks.register("lintAllDebug") {
    group = "verification"
    dependsOn(
            ":foo:app:lintAllDebug",
            ":bar:app:lintAllDebug",
            ":quax:app:lintAllDebug"
    )
}
so my original snippet, this is a lifecycle task as well?
i thought lifecycle ones were build, test, check .. the built in ones
v
Yes, that 
lintAllDebug
is a lifecycle task too
build
and 
check
are also lifecycle tasks
test
is not a lifecycle task, it is a task that executes the tests
check
is the lifecycle task that depends on 
test
build
is the lifecycle task that depends on 
check
and the lifecycle task 
assemble
v
this flips my mental model now 😄 build is a lifecycle because it just declares 
depeondsOn assemble, check
, check just 
depeondsOn test
so now..how do people plug them selves into check? I'm seeing they do, say in 
detekt
, the static analysis plugin
oookay I know
Copy code
task.named("check") {
   dependsOn detektStuff
}
right?
👍 1
v
Copy code
tasks.check {
    dependOn(detekt)
}
v
hmm but they see check..how? is it special? or can I do 
tasks.foo { .. }
as well?
in order for 
named
to see it, it has to be a task defined in that project, right? so it has to be special
v
If they want to hook into 
check
, they have to ensure the 
lifecycle-base
plugin is applied. If they apply any built-in plugin chances are high the 
lifecycle-base
plugin is also applied. If not, they can also apply the 
lifecycle-base
plugin directly. After doing so, the 
check
task is registered and available. If it is in the context of a Kotlin DSL build script or precompiled Kotlin DSL script plugin, then also the 
tasks.check
accessor is available. If not, the the 
tasks.named
you used is the right way to go, or 
val check by tasks.existing { ... }
if still in Kotlin.
in order for 
named
to see it, it has to be a task defined in that project, right?
Yes and no. In order for 
named
to see a task, the task has to be registered already the moment you call 
named
, for example by applying the 
lifecycle-base
(directly or transitively) before doing the call. If the task is going to be registered later or you want to configure it if it is registered already or also later, you can do 
tasks.named { it == "foo") }.configureEach { ... }
for example
v
okay so 
lifecycle-base
registers a no-op 
check
tasks to which people can plug into and then global agregator can just pick up all check tasks by name 
./gradlew check
is this right? (I mean the empty check task bit, I just inferred it)
v
Yes
v
gotcha, great! a lightbulb went off
thank you for your time!
v
You're welcome 🙂
v
Btw I've done all that, and it works well -- but I noticed my config time go up, by like 50% .. is that normal? I have like 400 modules -- so it went up from 10 to 15s
v
I have no idea, maybe you have to consult the 
gradle-profiler
to see where the time is spent. 🤷‍♂️
v
ill do that thanks
👌 1
Hi, if I might follow up on yesterday, so this is what I have
Copy code
// Root

subprojects {
    tasks.register("testAllDebugUnitTest") {
        group = "verification"
        // Lifecycle task
    }
    tasks.register("testAllReleaseUnitTest") {
        group = "verification"
        // Lifecycle task
    }
}

tasks.register("checkAllDebug") {
    group = "verification"
    dependsOn(
            ":foo:app:assemblePreprodDebug",
            ":foo:app:assembleProdDebug",
            ":bar:app:assemblePreprodDebug",
            ":bar:app:assembleProdDebug",
            subprojects.collect { "${it.path}:testAllDebugUnitTest" }
    )
}
I'm using 
checkAllDebug
to run checks over all of apps in the build, as a basic CICD check of every commit. Works well. However, when I'm building android app release, that's heavy and I build it separately per app (
:foo:app:buildAllRelease
)
Copy code
// Foo app

tasks.named("testAllDebugUnitTest") {
    dependsOn(
            "testPreprodDebugUnitTest",
            "testProdDebugUnitTest",
    )
}

tasks.named("testAllReleaseUnitTest") {
    dependsOn(
            "testPreprodReleaseUnitTest",
            "testProdReleaseUnitTest",
    )
}

tasks.register("buildAllRelease") {
    group = "build"

    dependsOn(
            ":clean",
            ..
            "testAllReleaseUnitTest",
            "assembleRelease"
    )
}
But now 
"testAllReleaseUnitTest",
obviously references just this project's task, not all of them. So I try the same pattern
Copy code
tasks.named("testAllReleaseUnitTest") {
    dependsOn(
            subprojects.collect { "${it.path}:testAllReleaseUnitTest" } <-------------------------
            "testPreprodReleaseUnitTest",
            "testProdReleaseUnitTest",
    )
}
But it returns empty list -- any idea why?
I mean I think I know why, because gradle project wise it has no children - but how can I make it so? Or maybe different technique Workaround I can see is to definfe the 
testAllReleaseUnitTest
in root and call that (
:testAllReleaseUnitTest
), but that will obviously call all the release tasks everywhere, not just that given app's - or should I just eat it?
v
That the
Copy code
subprojects {
    tasks.register
is a no-no you are aware, yes?
I mean I think I know why, because gradle project wise it has no children
Exactly
but how can I make it so?
You define in the settings script how your project tree looks like But - as you only share half of the information, so guessing here - you probably want to "call" that task on all projects the app project depends on. If that is the case, you should probably totally change the approach. Have an outgoing variant in the projects. (iirc it can have no artifacts just fine) Make sure the outgoing variant has task dependencies on the tasks you want to depend on. Use an artifact view on 
compileClasspath
or 
runtimeClasspath
or whatever is appropriate, with a component filter that matches only project dependencies, and requesting attributes so that this variant is matched. This way you actually get the originally requested logic of "trigger the task in the projects that have it and ignore all other projects", as artifact views by default ignore dependencies that do not provide the requested variant.
In the root project you could have a separate configuration where you depend on all projects, or all apps, and there use the same artifact view approach.
This setup is a bit more involved, but will probably get you exactly where you wanted to land at. 🙂
v
is a no-no you are aware, yes?
yes I ommited that
you probably want to "call" that task on all projects the app project depends on.
yes exactly - 2 modes, light check over everything (already works as per yesterday's chat) & a heavy check app specific (it would take like 3x the time to run it over everything, so unnecessary), i.e. to scope it down to just foo app's subgraph
Have an outgoing variant in the projects.
Do you mean this? https://docs.gradle.org/current/userguide/variant_attributes.html I'm only familiar with variants from android
v
Yes, that
v
somehow it feels im reinventing the wheel though
v
No, not really 🙂
v
what about telling gradle what the subprojects for given project are? would that break down if I have multiple apps that depend on the same module?
v
Yes
As I said, you define how the tree looks in your settings script
But I don't think you can have a project as subproject of two projects
That also does not make sense
As what you have is not a subproject, but a dependency
And to get something from those, you use the artifact view as I described
v
btw how would I define the nesting? via folders?
Copy code
:foo:app
:foo:data
:foo:ui
?
ie. 
./gradlew :foo:check
runs check over the 3? is that the idea?
v
What do you mean? With what you showed, you defined one project 
foo
with three subprojects 
app
, 
data
, and 
ui
.
If you do 
:foo:check
, then you execute the task 
check
of the project 
foo
If that task depends on the task 
check
in the other three, then it would be what you just said
v
im confused .. to be a project, is that just implied from folder structure - or do I need a build.gradle?
i mean i think of gradle as a graph of projects, so its kinda confusing to me that now dependency a project are different
since it sounds that project = ~build.gradle; and the hierarchy is then definied by nesting, i.e. implied from folder structure right? so 
foo
project might not even declare dependency on app, data, ui, yet be their parent project, because of folder strutcture?
v
Folders => irrelevant 
build.gradle
=> irrelevant
As I said a few times, you define in your settings script how the project tree looks like
Whether a project has a physical build script is irrelevant
And where you configure the project directories to be is irrelevant
If you do 
include("foo:bar:baz")
, you declare three projects with one call
A project 
foo
with a subproject 
bar
with a subproject 
baz
By default the project directories will be 
foo/
, 
foo/bar/
, and 
foo/bar/baz
, but you can configure it in the settings script to whatever you want.
v
As I said a few times, you define in your settings script how the project tree looks like
but how? this is what I'm missing. In settings.gradle I have just the
Copy code
rootProject.name = "Repo"
include ":foo:app"
include ":foo:data"
include ":foo:ui"
include ":bar:app"
include ":bar:data"
include ":bar:ui"
v
You can also do 
include("baz")
and configure its project directory to be 
foo/bar/baz
, then there are no 
foo
or 
bar
projects, even though you have the directory structure
but how? this is what I'm missing. In settings.gradle I have just the
I just explained it to you
With what you showed, you define 8 projects
v
if I were to define just
Copy code
include ":foo:app"
include ":foo:data"
include ":bar:app"
include ":bar:data"
that then means its 6 projects, foo being parent to app and data etc does that then mean I can now call say 
check
task on both fooapp an foodata, via just 1 call? or is the 
./gradlew check
-like syntax always going to be "everything inside build"
I'll assume you'll say no, and I need to define my own task that does that -- and now the 
subprojects.collect { .. }
should work -- but where do I define it, since in this case 
:foo
doenst have a build,.gradle -- I can just create a semi-empty build.gradle in the foo folder, without any plugins applied whatsoever?
OR, is the only way the 
variants
you mentioned?
v
The variant way is the clean way, especially when you then have a common lib-module. If foo and bar are completely separate, you should probably make them separate builds, then you also do not have any problems anymore. But well, given that setup, yes, you need to create your on task on 
foo
level that depends on the subprojects. Well, iirc, you could "go to" the foo folder and do 
../gradlew check
there to only search in the subtree of 
foo
, or you can do 
./gradlew -p foo check
, those should also do what you want iirc.
And yes, if you want to configure 
foo
, just create its build script that you omitted so far.
But actually you should not create your own 
check
task, but apply the 
lifecycle-base
plugin that adds the standard 
check
task as described yesterday.
v
question, if 
foo
is a project implicitly, regardless if it has a build script, that means it should see stuff 
lifecycle-base
applies, so 
check
, so I should be able to 
./gradlew :foo:check
right? yet I'm not .. 
Cannot locate tasks that match ':foo:check' as task 'check' not found in project ':foo.
v
No, why should it?
If no-one applies the 
lifecycle-base
plugin, no-one applied the 
lifecycle-base
plugin, so the tasks it registers are not present.
v
Idk, I figured its autoapplied or something 😄
okay so project can have actually 0 behavior on it out of the box
👍 1
v
Why should it be auto-applied?
Gradle cannot guess what you want a project to look like
Until you tell it
v
Okay just to sum it up if I understand it -- I could create a foo level build.gradle and define a task there that calls 
subprojects.collect
but it would only work in foo's children, which would break if the settings looked like
Copy code
include ":foo:app"
include ":foo:data"
...
include ":base"
(where 
:foo:data
has a dependency on 
:base
) and therefore I need the 
variants
method gotcha, thanks!
v
Well, it would not "break", but you would miss the 
base
, yes. Of course you can also do
Copy code
include ":foo:app:data"
include ":foo:app:ui"
include ":bar:app:data"
include ":bar:app:ui"
And have the "aggregator task" on "app" level. You could even do this without changing the directory layout, by configuring the project direcotries in the settings script.
Btw. I hope for you those namings were just for sake of conversation here. If the projects really have the same name, you will get into way more serious trouble.
v
hmm yes its a simplification but why? I do have 
base
project inside both foo and bar "subtrees"
isnt the "full qualified" name what counts?
v
Nooot a good idea. One point is that the jars are only called 
base-1.2.3.jar
which is not really helpful and also prone to clash, but Gradle will also get confused in some situation afair.
Well, iirc it might depend on the 
group
too, so it could be that it is not problem if they have different group. But if group and name is the same, there will be problems.
v
hmm but I do like the folder structure like that 
:foo:foo-base
seems rather redundant
v
You can keep the folder structure, you just need to rename the projects
v
I do remember java folks throwing everything into root (albeit hyphenated) and I hated that, you open up a repo and 500 folders..
ookay 😄
v
Yep, the problems come if GAV are the same, as that is the identity of a project in gradle.
GAV = groupartifactversion so groupprojectnameversion
v
I've never set a group on internal library module 😄
v
Yeah, then you can easily get into trouble
v
okay ill look into it but I dont see any problems currently; and we do for sure overload the names
i mean 
api
.. thats like 200 of those 😄 
foo:api
, 
bar:api
, ...
v
Really nooot a good idea, the catastrophe is pre-ordered 🙂
v
how would that manifest at all? build time fail?
v
If you are lucky
And then you might be majorly confused about it, read the issue I linked you to
But it could also just use the wrong one if it happens to still compile because the used APIs are the same but use the wrong implementation then and so on.
v
I guess android is saving me somehow
v
Imagine you have two sons and name them both John. If you now call for John, you never know which one will respond.
v
but I never call them John, I always call them 
:foo:John
😄 or 
:bar:John
i.e. I never have 2 johns under foo
v
No, you don't
Their parent projects are not part of their identity
It is group, artifactname, and version
v
wellp, that's news
v
... to you 🙂
Read the issue I linked you to, I don't do that for fun 😉
v
I know you're not lying 😄 I'm just talking but then referencing the module 
:foo:foo-api
looks weird (sure I can have file system be different, I'm aware; but the reference will have to be like that, right?) do people really do it like that? or should I have just 
:foo-api
gradlew wise and then file system wise 
foo/api
?
v
but then referencing the module 
:foo:foo-api
looks weird
Well, some kids do look weird 😄
do people really do it like that?
Ask the people 🙂
or should I have just 
:foo-api
gradlew wise and then file system wise 
foo/api
?
Whatever pleases you most and works for you.
As I said, you could also make 
foo
a standalone build and 
bar
a standalone build and have a composite build that combines them for joint-checking or other stuff.
v
probably wont help, as the 
api/impl
is a pattern we use everywhere, multiple times even within 
foo
, so everything has it basically
but simply adding a group should do the trick, right? maybe I can stick that into convention plugin?
but I'd recon its android that fixes this for me, no? dexer crunches all the jars.. but okay makes no sense as you said jars will have the same name..but maybe it references them via a path with do is unique
v
I have no idea what Android does,I don't do Android
2 Views
Open in Slack
PreviousNext
Powered by