This message was deleted.
# develocity
s
This message was deleted.
t
I'd guess that its doesn't show up there because it uses the worker API so even though it was 20s the bulk of it wasn't in the task action function.
IE it was probably correct at some time in the past but as Gradle APIs and implementation has evolved it started to make less sense
w
The sub-parts of the Duration don’t capture everything that happens for executing a task. For example they don’t capture the time it takes for fingerprinting the outputs after the task executed. Since this is a regular Copy task, I don’t see the worker API being involved or a lot of dependency downloads happen. Moreover, since the action execution is quite quick it seems like it isn’t copying a huge amount of files. What is the destination directory of the
Copy
task? Does it contain a lot of files? When fingerprinting the outputs Gradle will capture everything in the destination directory not only the copied files. So anything else that ends up there that might be expensive reading might make the task slow. Finally, can you share what is running in parallel to this task on the timeline? I wonder if something else is affecting this task. All in all, this is very strange and I didn’t see something like this before.
n
This is a task that copies all the jar dependencies to a folder so they can then be used by a docker build. Here is the definition:
Copy code
val dockerRuntimeLibsDir = "${layout.buildDirectory.get()}/dockerRuntimeLibs"
val deleteDockerRuntimeLibs by registering(Delete::class) {
        delete.add(dockerRuntimeLibsDir)
    }
val copyDockerRuntimeLibs by registering(Copy::class) {
        into(dockerRuntimeLibsDir)
        from(configurations.runtimeClasspath)
        dependsOn(deleteDockerRuntimeLibs)
    }
I guess fingerprinting all the jars is slow after the copy is finished.
w
Is there anything else in the
dockerRuntimeLibs
directory? 20s for fingerprinting a few JARs seems to be very high.
n
as you can see the directory is deleted before the copy so it is empty
w
Anything that runs between the delete and the end of the copy task? It looks like something is happening in parallel.
n
come to think of it this shouldn’t be cached.. the jars are going to be different on every run (we embed a txt file with a version number in it). If I say
cachIf { false}
will it fingerprint?
The build is highly parallel so there is a LOT running in parallel
w
Anything running in parallel that accesses the same or a parent directory to the
dockerRuntimeLibs
?
n
the parent directory is the
build
so almost certainly something might be running yes. We have configuration cache so tasks in the same module can run in parallel.
w
Can you reproduce that? if yes, then I suggest you capture a Chrome trace with gradle-profiler for this build run and then I hope you should be able to figure out what is going on.
n
I will struggle to reproduce that. It only happens in CI. Not sure how often it happens.
What is a Chrome trace?
n
neat.. didn’t know about that.
Do you know if I disable caching for the copy task if that will disable the fingerprinting?
w
It won’t. What you can do is to use
Task.doNotTrackState()
. That switches off all the state tracking for a task, including the input and output fingerprinting.
n
Perfect! that is probably what I should do!
w
I am not convinced yet that output fingerprinting is the culprit. Though I don’t have many other ideas what could happen that is not captured within the existing parts of the “Duration” dropdown.
n
Any plans to enhance the duration breakdown in future releases?
w
We have some plans, though not sure yet when we’ll implement them.
Which Gradle version are you using BTW?
n
8.2.1
w
Did you see the problem before that version as well?
n
I can have a dig around… I only noticed it by chance
which version are you interested in?
w
Happy to help.
I would be interested if you can find out how often this happens. If it is a one-off thing then it might also be machine related.
n
I think it is pretty consistent. Although the example above is an extreme case.
Here is a gradle
8.1
version
w
Regarding the version: I don’t know of any regression/improvement that could be responsible for this behavior. I was wondering if it was something new introduced in Gradle 8.2.1 or something that happens on older Gradle versions as well.
n
Unfortunately it seems Gradle enterprise is only storing 90 days worth of builds and so my data only goes back to version 8.1
I will try
Copy code
outputs.cacheIf { false }
        doNotTrackState("It is going to be different on every run because of version.txt")
And report back
w
The
outputs.cacheIf { false }
won’t have an effect because the task isn’t cacheable anyway.
n
oh I see….thanks
let me see if the other thing makes a difference…
Do I need
Copy code
outputs.upToDateWhen { false }
w
doNotTrackState
implies all the other things. That is enough.
n
I think that has made the difference
This is locally, will now run in CI to confirm.
It has definitely made a difference. I found a couple more tasks I can optimize this way. Thank you both for your help.
w
This is not an optimization. This will hurt local devs if they run this task as part of an incremental build. It would be better to fix the underlying problem why accessing the outputs takes so long.
n
this task will never be upToDate because of the way we create the jars… It is very rarely run locally anyway. It is only run in CI.
I have another question. I disabled the tracking of state for processTestResources and yet fingerprinting still happened:
w
Fingerprinting time includes a few things: https://github.com/gradle/gradle/blob/b7717d970b4bbb5543059bfbea865138bb017128/sub[…]org/gradle/internal/service/scopes/ExecutionGradleServices.java Maybe in this case it is checking whether the sources are empty? Why did you try to mark
processTestResources
with
doNotTrackState()
?
What is your max-workers setting on CI?
n
as you can see from the screenshot it takes 12seconds to fingerprint the inputs which dominates the runtime of this task. Given that the task is not cacheable I don’t see why fingerprinting needs to run. I am not aware of a max-workers setting so it must be what the default is. There are 20 CPUs in CI.
d
Given that the task is not cacheable I don’t see why fingerprinting needs to run
Fingerprinting is also required for
UP-TO-DATE
checks, so by using
doNotTrackState()
you're forcing the task to always run, even if there's been no change. This is bad for incremental builds.
n
As you can see task execution is 0.062 seconds and fingerprinting is 12 seconds. I'd say running every time would be preferable if those numbers are to be believed
a
Could you maybe create a small reproducer or give some more information what what kind of files you process (how many files, what is the size of files, what type of files, what filter you use in processTestResources etc)? Also what kind of files you copy with
copyDockerRuntimeLibs
? Are these just jars, and how many of them, what size etc?
n
The only thing that might be causing a problem with processTestResources is that I have defined one central location for testResources which is used across all modules. So all modules are trying to access the same folder concurrently.
copyDockerRuntimeLibs
is just jars. You can see the code for it further up in the thread. The size and jars vary from module to module. The biggest jar is a transitive dependancy of kafka
rocksdbjni-7.1.2.jar
which is 52MB. There are 170-250 jars that get copied depending on the module