hello! struggling to get my artifact transform to ...
# plugin-development
f
hello! struggling to get my artifact transform to run. I’ve set up a configuration with a particular attribute, I’ve registered the transform as per the docs. I’m resolving some artifact from something that has gradle-module-metadata. The attribute on the variant from this producer, has the same type as the
from
in the transform. What I don’t understand is: •
Incompatible because this component declares attribute 'org.gradle.usage' with value '<thing to convert>' and the consumer needed attribute 'org.gradle.usage' with value '<desired>'
• This is gradle 8.4. I’m getting a
NoMatchingConfigurationSelectionException
and I’ve traced through the code for the failure path. It sounds like the configuration at this point should not have an attribute on it so it can pass the
isMatching
check, but I’m not sure when I should put the attribute on it? By the time it reaches
AttributeMatchingConfigurationSelector.selectVariantsUsingAttributeMatching
is it too late to do artifact transforms? • In a separate instance of me using artifact transforms successfully, it’s a very similar setup (albeit not with GMM) but using the
artifactType
attribute converting from
tgz
to
directories
. Why is it okay for the
Configuration
to have that attribute and the transform be active, but in the above, it refuses to match the attribute? • Should I be forcing the issue with an
ArtifactView
?
some more context: in my above example, I’m failing at the
DependencyGraphBuilder#traverseGraph
step in
DependencyGraphBuilder#resolve
and if I’ve traced correctly, the artifact transforms happen in the
DependencyGraphBuilder#assembleResult
basically, what I want to be able to do is: • define a configuration that requires a particular attribute, • different repositories will return 1 or 2 different values for the above attributes and won’t match directly. • my artifact transform should take it from whatever attribute gets returned from the repository GMM and transforms that to the desired attribute specified in the
configuration
v
Actually it's a bit of a mess, for example
artifactType
is an attribute on artifact, not on the variant so that's a significant difference in your two cases. Can you maybe knit an MCVE that shows your non-working try?
f
sorry was quite late in the day for me when you replied. I’ve made a repro here: https://github.com/felixdesouza/gradle-artifact-transform-reproducer build scan: https://scans.gradle.com/s/vlbvubqqo7zwm different cases: https://github.com/felixdesouza/gradle-artifact-transform-reproducer/blob/kotlin-dsl/build.gradle.kts failing github actions: https://github.com/felixdesouza/gradle-artifact-transform-reproducer/actions/runs/9161382659/job/25186099784 so I played around a bit more, and it seems that if I don’t specify any attributes on the
Configuration
, and instead make an
artifactView
with the attribute I want, then I’ll get the transform to run. I’ve added that as a case above.
I suppose what’s confusing is that lots of things have attributes, and it’s not immediately clear how they contribute to resolution, and the artifact transform docs only work because of the artifact type, not artifact transformations in general. From what I can surmise, attributes on `Configuration`s are effectively for exact matching/filtering and if artifactType is specified then it does a transform where possible (as long as you don’t require the attribute you actually want).
In addition, I was hoping to use the whole
RepositoryContentDescriptor#onlyForAttribute
and have a dependency specify a an attribute which would mean it would try to resolve it against a particular repository, but that doesn’t sound like it’s doable. What is the most convenient way of specifying that a dependency should be resolved from one repository. I could do the fallback thing, but wanted things to fail if a dependency cannot be found in the repository it is “supposed” to be in, rather than resolving silently via fallback. Should I have multiple separate configurations and then combine them via
extendsFrom
?
v
@Jendrik Johannes please correct me if I got it wrong, as the attribute topic is always a mess to understand, and I'm also always not sure I fully get it right.
confusing is that lots of things have attributes, and it’s not immediately clear how they contribute to resolution
Yes, as I said, it is pretty messy. Configurations have attributes, artifacts have attributes. The artifact transforms are artifact transforms, not variant transforms. For selecting a variant, the attributes on the configuration level have to match and you cannot "transform" these. After a variant is successfully selected, an artifact transform and further transform the artifact. For
artifactType
it works on the configuration because it is a artifact-level attribute, so it does not contribute in selecting the variant. So despite the
artifactType
not matching, the variant can be selected, and then the artifact transform can transform the artifact to the intended state. With the artifact view, the configuration resolved the variant successfully as you did not request the usage attribute and not requesting an attribute is compatible to the attribute having any value. Then for the artifact view the transform can kick in to transform the attribute to the intended state.
Should I have multiple separate configurations and then combine them via
extendsFrom
?
If I got you right, this will not help unless you declare the wanted attribute on the dependency instead of the configuration, but then you could right away do it on the "one" configuration too, because
extendsFrom
just means that it is like you declared the dependencies on the extending configutaino like you declared them on the extended configuration. But the attributes you declare on the extended configuration are not relevant when resolving the extending configuration. For that only the attributes declared on the extending configuration (i.e. on the configuration you are actually resolving) are relevant.
f
Interesting, I think in the general sense that distinction makes a lot more sense. Thank you! I think the other point of confusion for me is that the configuration in the docs makes it such that the consuming
Configuration
has an attribute request of
minified=true
How does this not fail when trying to resolve via variant attribute matching? As this doesn’t fall into the above cases you outlined? i.e. if I request an attribute and the producer doesn’t have it at all (vs it does but it’s different), what is supposed to happen here? Would it fail harder if the upstream repository used gradle module metadata or something else? Effectively, when is putting an artifact on a
Configuration
going to contribute to artifact attribute matching vs variant attribute matching? Having
artifactType
automatically work is probably what has caused me the most confusion i.e. it would make more sense that to do any sort of transform you have to use an
ArtifactView
. it’s a bit more cumbersome but definitely better drives the point home. But alas, we are here now 😅 . IMO the docs should focus on the
ArtifactView
approach in addition to the existing, because the above
unless you declare the wanted attribute on the dependency instead of the configuration
is this for variant attribute matching or artifact attribute matching? if I’ve understood correctly. (Assuming it’s also variant attribute matching). I should be able to add an attribute when adding a dependency. That should then be checked against the producing attributes and I could get the “mutual exclusion” I care about that way? FWIW, I control both consumer and producer.
v
How does this not fail when trying to resolve via variant attribute matching?
minified
is not an attribute on the configuration You add it with
artifactTypes...
with value
false
to all
jar
attributes and thus it is like
artifactType
an attribute for the artifact, not the variant, so the artifact transform can do its work after the intended variant was resolved.
is this for variant attribute matching or artifact attribute matching?
I'm not fully sure, but I think for variant matching, because first the variant has to match before the artifact can be transformed. And setting it on the dependency should be like setting it on the configuration just that it is only for that dependency and also carried over to extending configurations as it is delcared on the dependency.
f
So just to be clear: (copied from the docs)
Copy code
configurations.all {
    afterEvaluate {
        if (isCanBeResolved) {
            attributes.attribute(minified, true) 
        }
    }
}
now has a different meaning because we’ve added
minified
as an attribute to an
artifactType
?
v
Did I mention it is a great mess? 😄
😅 1
Yes, by adding to the artifact type, it is an artifact-level attribute, not a variant-level attribute
So the variant can be found first and then the transform can transform the artifact accordingly
f
haha! thanks! I think I will play around with this
👌 1
I am back again! Lots of success with the
ArtifactView
. I remember you said that
dependency
attributes +
configuration
attributes are effectively combined and used for variant selection. I was hoping to use
RepositoryContentDescriptor#onlyForAttribute
but that only considers attributes from the configuration. I was spelunking through the code, and it didn’t seem like there’s a reason to not add this prefiltering. It shouldn’t change correctness, but it would avoid roundtrips where you know that the repository in question does not have the dependency. As I don’t expose a raw
DependencyHandler
block to the plugin consumers, I control what gets added to the configuration. I was hoping to have in my dsl something along the lines of:
Copy code
myBlock {
  product ('some-product-group:some-product') {
    local()
  } 
}
In my ideal world, this sets a property, which would be transformed into an attribute that I add to the
some-product-group:some-product
Dependency
object. That doesn’t work as aforementioned. The other option is just calling
RepositoryContentDescriptor#includeModule
and
RepositoyrContentDescriptor#excludeModule
on the respective repositories. But it’s awkward because properties don’t have any concept of a
configure
. I could do this with an
afterEvaluate
but seems a bit meh. Do you have any ideas what would be decent to do here? I don’t think I want to have separate configurations because I don’t believe the resolution will be the same even with the same set of initial constraints.
v
No idea about
onlyForAttribute
, never used it. But from the description I'd say it should work for attributes declared on the configuration and on the dependency alike. If it does not, you might have discovered a bug that you should knit into a reproducer and report to Gradle.
As I don’t expose a raw
DependencyHandler
block to the plugin consumers, I control what gets added to the configuration.
You might be interested in this that is added in 8.8. Of course it is incubating and also only works with 8.8+, but it might be interesting or something for the future: https://docs.gradle.org/8.8-rc-1/userguide/implementing_gradle_plugins_binary.html#custom_dependencies_blocks
The other option is just calling
RepositoryContentDescriptor#includeModule
and
RepositoyrContentDescriptor#excludeModule
on the respective repositories.
Maybe better use
exclusiveContent
instead? https://docs.gradle.org/current/userguide/declaring_repositories.html#declaring_content_exclusively_found_in_one_repository
I could do this with an
afterEvaluate
but seems a bit meh.
Not just "meh", the main effect of
afterEvaluate
, especially in plugins, is to add ordering problems, timing problems, and race conditions.
Do you have any ideas what would be decent to do here?
Not really, as I didn't fully get what you try, sorry. Maybe some MCVE could bring clarity.
f
Not just “meh”, the main effect of
afterEvaluate
, especially in plugins, is to add ordering problems, timing problems, and race conditions.
haha yeah, I try to remove it where possible in our plugins, I can’t count how often it has caused problems.
You might be interested in this that is added in 8.8.
Of course it is incubating and also only works with 8.8+, but it might be interesting or something for the future: https://docs.gradle.org/8.8-rc-1/userguide/implementing_gradle_plugins_binary.html#custom_dependencies_blocks
I saw this in the release notes, it’s very neat. We are struggling with our gradle 8 migration across the board, so it’ll be a while before we can roll our plugins to use the block. But it is very neat from the look of things. Yeah, I will put up a reproducer, thanks!
👌 1