This message was deleted.

struggling with the exact same thing right now: https://gradle-community.slack.com/archives/CAHSN3LDN/p1690277073170139 maybe we can compare notes?

Great let me take a look .

unfortunately not, I can't share the codebase involved here 😞 I could look into setting up a reproducer, but that might take a while

Anything helps. Not sure how much sense my project makes to you, but I think that I need to move the variant declaration stuff to my buildSrc (“included build build-logic”) or something?

shouldn't you just match up the required capability with the registered one? https://github.com/xtclang/xvm/blob/build-aggregator/lib_collections/build.gradle.kts#L26

Hmm yes but it doesn’t resolve for some reason. I don’t think “producer” is right

https://github.com/xtclang/xvm/blob/build-aggregator/lib_ecstasy/build.gradle.kts#L32 <-- that's the capability you created right? so you should match that?

Yes.. I am taking a look at that right now, and that is the example I was wondering about. Where does “-producer” come in ? However, now I’ve added an xtcLibraries task that depends on the builds and returns the file set of modules, but a lot of the logic only seems to allow one destination file, including outgoingVariants

it's

producer-

, not

-producer

.

producer

is the producing module in this case, right? So in your case it's

lib_ecstacy-xtcFeature

. But given that you manually create the capability name, that's what you should require.

Thanks! I will push what I have so far but I need to go to lunch for a customer for around 1.5 h now 🙂

I'm reworking some of my setup, as it didn't seem to make much sense after having a more critical look. I'm now also trying the registerFeature approach, which seems more convenient. My producer side looks ok now. Still struggling to fix the consumer, but that might have more to do with unrelated gradle 7 -> 8 upgrade work. I'll keep you posted if I have a fully functional setup.

I agree. I have problems resolving a configuration that I define in the buildsrc(build -logic included build). While both projects find it and I add the outgoing artifacts correctly in the producer, the consumer project does not resolve it with “myConfig”(producer.module)… I guess the isCanBeResolvabe/consumable flags are messing things up

Didn't follow the whole conversation, but feature variants should also just work fine in included builds and even with published projects as long as GMM is published along. An example would for example be https://github.com/Vampire/setup-wsl/blob/master/gradle/dependency-updates-report-aggregation/src/main/kotlin/net/kautler/dependency-updates-report-aggregation.gradle.kts, but it is not a trivial one.

@Vampire Thank you for the link. Yes, it’s not trivial, I am afraid but it should give me some things to explore that I haven’t tried. Basically, the use case is, I have a compile task in my langauge plugin, that generates a new kind of binary artifacts (as far as gradle knows), .xtc files . The source set resolution works wonderfully and utilizes all logic it should, and, just as I hoped, just hooks into Gradle updated files, and otherwise returns cached results. So the input set is just lovely. It’s the concept of 1) understanding exactly how to make the rest of the world aware that my module creates these xtc files, so that other modules that contain other packages in the .x language can import the first module as dependency so that the plugin can pass the xtc binary paths along to the compile tasks as additional inputs, basically as compileClasspath artefacts would work in Java. The compile task (in the plugin that implements it) is basically just a Java exec that needs the paths to these files to generate a command line (and of course to make sure the first producer of the binaries is executed if these xtc files don’t exists). What I have trouble with is 1) the concept of resolving multiple files as output artefacts - some of these modules produce several xtc files, which are all artefacts, but it’s fine if a consumer just gets them all,. I don’t need any more fine grained control. and 2) these dependencies, due to later architectural needs, are all module / common build repo based because we want to use includedBuilds everywhere. For a “normal” gradle include, I could use something like implementation(project(“:producer”), xtcArtifactsProducedConfig) (or a custom depenency). But without the project reference , I am stuck for some reason … It does indeed look like this is what feature variants is intended for. I would love to see more examples of included build based runnable dependency stuff like this and more for custom artifacts in the gradle user guide. I have tried to really get my head around the gradle architecture, and I do get what you are trying to build and think it makes a lot of sense, given that the abstraction has to cover so many generic problems, but still can’t just “disappear” into “I am some kind of object”-land….

Yeah, referring to the configuration is just for referencing within one build, so no included builds and no external consumers. That's why it is the "simple" way. Custom attributes, or feature variants are exactly the way to share cross-build, be it included builds or completely separate builds. For multiple artifacts, you just add all of them to the outgoing configuration. If then someone depends on it, he gets all of them. If you are stuck, you could share your efforts or an MVCE, and I could maybe have a look when I'm back from vacation.

Thanks. I will try to get that working . Mostly just looking for a simple example that has two projects available as includedBuild, one producing custom artefacts, and one consuming them by declaring a dependency on them, however you now do that in the consumer included Build. I get that I cant’ refer to projects, so I need to create some kind of custom dependency that ensures the artefacts are generate or generates them, when it is resolved. A feature variant based example that does something like this has for some reason been quite hard to find, and I haven’t been able to get a perfect fit for it from trying to go from the manual snippets into my project code …..

Simple but complete example. Whether better to use a feature variant or pure attributes, and whether to use standard attributes or a custom attribute, and which attributes, and which values, ... highly depend on the concrete semantics and might not be easy to get right. For simplicity of the example I just used two standard attributes:

./bar/build.gradle.kts
import org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE
import org.gradle.api.attributes.Category.LIBRARY
import org.gradle.api.attributes.LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE

val producerTaskA by tasks.registering {
    val output = layout.buildDirectory.file("a.xtc")
    outputs.file(output).withPropertyName("output")
    doLast {
        output.get().asFile.writeText("A XTC content")
    }
}

val producerTaskB by tasks.registering {
    val output = layout.buildDirectory.file("b.xtc")
    outputs.file(output).withPropertyName("output")
    doLast {
        output.get().asFile.writeText("B XTC content")
    }
}

val producerConfiguration by configurations.registering {
    isCanBeConsumed = true
    isCanBeResolved = false
    outgoing.artifact(producerTaskA)
    outgoing.artifact(producerTaskB)
    attributes {
        attribute(CATEGORY_ATTRIBUTE, objects.named(LIBRARY))
        attribute(LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("xtc"))
    }
}

./bar/settings.gradle.kts
rootProject.name = "bar"

./build.gradle.kts
import org.gradle.api.attributes.Category.CATEGORY_ATTRIBUTE
import org.gradle.api.attributes.Category.LIBRARY
import org.gradle.api.attributes.LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE

val consumerConfiguration by configurations.creating {
    isCanBeConsumed = false
    isCanBeResolved = true
    attributes {
        attribute(CATEGORY_ATTRIBUTE, objects.named(LIBRARY))
        attribute(LIBRARY_ELEMENTS_ATTRIBUTE, objects.named("xtc"))
    }
}

dependencies {
    consumerConfiguration(":bar")
}

val foo by tasks.registering {
    inputs.files(consumerConfiguration)
    doLast {
        consumerConfiguration.files.forEach {
            println()
            println(it.name)
            println(it.readText())
        }
    }
}

./settings.gradle.kts
rootProject.name = "foo"
includeBuild("bar")

Output of `gw foo`:

> Task :bar:producerTaskB
> Task :bar:producerTaskA

> Task :foo

a.xtc
A XTC content

b.xtc
B XTC content

Thanks. That does indeed seem to work. But I think something is wrong with the included builds that fails to resolve stuff from my side. Either just declaring a usage attribute OR using your example attributes above - thanks a lot for that example btw - still fail to resolve.

abstract archive task should be fine. Any task that declares output files properly should be fine. Can you provide an MCVE?

Yes, I have very few cycles this week and some other priorities, but I will try to boil this down to an MCVE during the weekend! Thanks!

I wonder if this has to do with my category attributes. Trying to figure out what the “implementation” configuration has as attributes, but it’s kind of hard to find it in the gradle source code.

Probably none,

implementation

is not for resolving, so attributes should be irrelevant. You want to have a look at

runtimeClasspath

or

compileClasspath

.

Yes - adding the inputs.files(consumerConfiguration) to my compile task in the consumer did indeed force the producer to be built.

Of course things are only built when needed. Gradle is very good at avoiding unnecessary work. But for that it of course needs to know what are input of a task to then also derive task dependencies from that. If you just resolve your configuration at task execution time, it is too late and Gradle did not know it needed to run something else first. By declaring the configuration as input to the task, it knows that the contents are input for the task and thus can also trigger the necessary tasks.

The input thing seemed indeed to be the missing piece. I’ll try to get my hacky code back into best practice-Gradle, check that it resolves my dependecies and then go back to the includedBuild world.. Thanks a lot! you have been a huge help so far

I wonder if this has to do with my category attributes. Trying to figure out what the “implementation” configuration has as attributes, but it’s kind of hard to find it in the gradle source code.

Btw. have a look at the

resolvableConfigurations

task. It lists all resolvable configurations and their attributes.

Yes I’ve studied these in detail, and experimented with several configs. Just putting a usage in there, was also something that confused me, as then suddenly transitive dependencies (e.g. java libraries) that the producer depends on, suddenly did not resolve in the consumer, even though the consumer doesn’t know or care about these, just about its buddy the producer

Yeah, well, you might then want to make the configuration or the dependency non-transitive if you don't care about the dependencies. Or if you want the xtc stuff from the dependencies if present, you might use a lenient artifact view which does not fail if something cannot be resolved but just gives you what it found.

Understood. But in this case, the producer depends on a java library, which it uses for some tasks, that do not become inuputs for any other project. Yet the consumer complains that it cant’ find it when resolving the consumer. Not sure why it picked up an implementation(libs.standalone.java.library) in the consumer at all through my xtcModule (consumer configuration),but with the attributes from your example, category attribute “library” and library elements “xtc”, it didn’t exibit this behavior

Yeah, well, as I said, actually defining the right attributes and values that should be used is the hardest part in the whole topic. 🙂

Indeed. Here I find that I have very little documentation or even examples to go on. Is there a reference you can recommend? The user guide basically tells me exactly that - that it’s hard, and not exactly what all of them do, or what behavior a custom attribute property will affect

None I know of

One more question - if my producer task creates one to many outputs, that are not known at configuration times - can I still configure them as n outgoing artifacts (where n > 1) in the configuration phase in the producer configuration? There is an @outputfiles annotation, but I don’t know how many they will be until the task is resolved and run. Is the only way I can do this to specify the @outputdirectory where the producer task will place them?

Not sure, TIAS 🙂

I guess I will. Getting rid of the intermediate stage of zipping up this artbitrary number of artifacts to treat it as a resolveable file, is something I would like to avoid if I can

Might be, that you need to configure a directory then, yes.

It does indeed seem to work. That is lovely. (not trying to take out anything advance here, I have been tripped up before)

The latter

@Vampire I will consult my work logbook when this is fininished, and I’d be happy to spend any spare cycles requesting or adding functionality. A pull request with docs with a couple of explaining paragraphs in places, is certainly not too much to contribute if anyone will proof read and get it merged.

Can take some time nowadays as Gradle folks are short on resources, but generally contributions are still welcome and processed, just a bit delayed.