Is the iteration order of a

FileCollection

predictable?

It's JavaDoc says

A file collection is can be used to define a classpath

and as classpaths are ordering sensitive, I'd very much hope it is predictable / consistent. If it were not, I guess that would also be pretty bad for up-to-dateness and cacheability.

Right 👍 I’m guessing the order is implementation details, though, right?

Could be considered like that I guess, yes. But if you for example have a configurable file collection and add things to it, it should hopefully preserve insertion order, or you can hardly construct a classpath that is order-sensitive (even if it is more or less bad practice). But if you just have a

FileCollection

and don't know how it is built-up / implemented, I'd expect consistent ordering on iteration, but if you don't know how it was built / implemented, you can hardly predict the order, yes.

I see, thanks!

Yeah, listing files on the filesystem could indeed different. Please link the issue here, I'm curious whether they agree it is a bug or works-as-designed. 🙂

Is the order even relevant to input snapshotting?

I guess it depends on the normalization. If it is

@Classpath

or

@CompileClasspath

, order should definitely be considered.

Bug is there: https://github.com/gradle/gradle/issues/29321

You mean you get a cache-hit even if the order in a

@Classpath

property is different? That sounds wrong to me

I didn’t go as far as testing end to end but the cache key is the same and the iterating order is not

Fwiw, looks like you're iterating a FileTree actually, not just a FileCollection (and even more specifically, a SourceDirectoryTree). AFAICT those are indeed platform-dependent: the srcDirs are iterated in the order they're added, but then each is transformed to a FileTree whose iteration order is platform-dependent. And AFAICT this would still be compatible with

@Classpath

in the same way that

-classpath "somedir/*"

passed to the JVM is patform-dependent too.

I did a quick test with this:

val a by configurations.registering { isTransitive = false }
val b by configurations.registering { isTransitive = false }
dependencies {
    a("commons-io:commons-io:2.4")
    b("commons-lang:commons-lang:2.4")
}
@CacheableTask
abstract class Foo : DefaultTask() {
    @get:Classpath
    abstract val classpath: ConfigurableFileCollection

    init {
        outputs.upToDateWhen { true }
    }

    @TaskAction
    fun foo() {
        classpath.forEach { println(it) }
    }
}
val foo by tasks.registering(Foo::class) {
    classpath.from(a)
    classpath.from(b)
}
val bar by tasks.registering(Foo::class) {
    classpath.from(b)
    classpath.from(a)
}

which results in

Resolve mutations for :foo (Thread[included builds,5,main]) started.
:foo (Thread[included builds,5,main]) started.

> Task :foo
Appending implementation to build cache key: Build_gradle$Foo_Decorated@b553d57856188a37e9c153a99e17afac
Appending additional implementation to build cache key: Build_gradle$Foo_Decorated@b553d57856188a37e9c153a99e17afac
Appending input file fingerprints for 'classpath' to build cache key: 80b864fcce8ee35d1fbbd1c341205786 - CLASSPATH{~\.gradle\caches\modules-2\files-2.1\commons-io\commons-io\2.4\b1b6ea3b7e4aa4f492509a4952029cd8e48019ad\commons-io-2.4.jar=IGNORED / d472b1d1d088ae048cd96c7c0939fbf5, ~\.gradle\caches\modules-2\files-2.1\commons-lang\commons-lang\2.4\16313e02a793435009f1e458fa4af5d879f6fb11\commons-lang-2.4.jar=IGNORED / aece18b33613844514d59ddfe00c2f1c}
Non-cacheable because No outputs declared [NO_OUTPUTS_DECLARED]
Caching disabled for task ':foo' because:
  No outputs declared
Task ':foo' is not up-to-date because:
  No history is available.
~\.gradle\caches\modules-2\files-2.1\commons-io\commons-io\2.4\b1b6ea3b7e4aa4f492509a4952029cd8e48019ad\commons-io-2.4.jar
~\.gradle\caches\modules-2\files-2.1\commons-lang\commons-lang\2.4\16313e02a793435009f1e458fa4af5d879f6fb11\commons-lang-2.4.jar
Resolve mutations for :bar (Thread[included builds,5,main]) started.
:bar (Thread[included builds,5,main]) started.

> Task :bar
Appending implementation to build cache key: Build_gradle$Foo_Decorated@b553d57856188a37e9c153a99e17afac
Appending additional implementation to build cache key: Build_gradle$Foo_Decorated@b553d57856188a37e9c153a99e17afac
Appending input file fingerprints for 'classpath' to build cache key: 45a7937557734f93d73fa600c2e4cb16 - CLASSPATH{~\.gradle\caches\modules-2\files-2.1\commons-lang\commons-lang\2.4\16313e02a793435009f1e458fa4af5d879f6fb11\commons-lang-2.4.jar=IGNORED / aece18b33613844514d59ddfe00c2f1c, ~\.gradle\caches\modules-2\files-2.1\commons-io\commons-io\2.4\b1b6ea3b7e4aa4f492509a4952029cd8e48019ad\commons-io-2.4.jar=IGNORED / d472b1d1d088ae048cd96c7c0939fbf5}
Non-cacheable because No outputs declared [NO_OUTPUTS_DECLARED]
Caching disabled for task ':bar' because:
  No outputs declared
Task ':bar' is not up-to-date because:
  No history is available.
~\.gradle\caches\modules-2\files-2.1\commons-lang\commons-lang\2.4\16313e02a793435009f1e458fa4af5d879f6fb11\commons-lang-2.4.jar
~\.gradle\caches\modules-2\files-2.1\commons-io\commons-io\2.4\b1b6ea3b7e4aa4f492509a4952029cd8e48019ad\commons-io-2.4.jar

So different order here result in different cache keys. Maybe it is handled differently or has a bug when it is coming from one source with different order, so if you list files in an OS directory and it then mitigates the different order. But even if so, if later the iteration order is different, it should probably not mitigate the different order there.

The docs here say "The order of the files in a FileTree is not stable, even on a single computer"

Ah, yeah, right. FileTree indeed do not have a guaranteed order.

Means I have to sort my

FileCollection

because I can’t make any assumptions how it was created

Fwiw, JavaCompile task redeclares its inputs to make sure the sources (inherited from SourceTask, as a FileTree) are _stable_: https://github.com/gradle/gradle/blob/master/platforms/jvm/language-java/src/main/java/org/gradle/api/tasks/compile/JavaCompile.java (same goes for GroovyCompile, but not e.g. Javadoc 🤷)

Interesting. I wonder why. The sources should not be order-sensitive and are

@InputFiles

. Do you know why that was necessary?

private final FileCollection stableSources = getProject().files((Callable<FileTree>) this::getSource);

Wait, how does that ensure stability? Is

files()

going to sort by path or something like this?

I think the

stable

does not refer to the order of the files, but about the

FileCollection

instance.

Oh I see

But

JavaCompile#getSource()

calls

super.getSource()

So much stability

And that calls

sourceFiles.getAsFileTree().matching(patternSet)