I want to use artifact transform to transform a de...
# plugin-development
y
I want to use artifact transform to transform a dependency's classes and its sources ,but gradle only pass the classes jar to my transformation, so how can i ask gradle to give the sources jar
project.getDependencies().registerTransform(
AccessTransform.class,
parameters -> {
parameters.parameters(p -> {
p.getAccessTransformerFiles().from(extension.getAccessTransformerFiles());
});
parameters.getFrom().attribute(
ModAccessTransformExtension.TRANSFORM_ACCESS,
false
).attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, ArtifactTypeDefinition.JAR_TYPE);
parameters.getTo().attribute(
ModAccessTransformExtension.TRANSFORM_ACCESS,
true
).attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, ArtifactTypeDefinition.JAR_TYPE);
}
);
m
You can try to use the
org.gradle.docstype
and match on
"sources"
. Never used that myself but looks like this is the attribute that Gradle uses to indicate sources variants
y
i tried this, but it doesn't work
Copy code
project.getDependencies().getArtifactTypes().named(ArtifactTypeDefinition.JAR_TYPE, type -> {
    type.getAttributes().attribute(ModAccessTransformExtension.TRANSFORM_ACCESS, false);
});

var objects = project.getObjects();
project.getDependencies().registerTransform(
        AccessTransform.class,
        spec -> {
            spec.parameters(p -> {
                p.getAccessTransformerFiles().from(extension.getAccessTransformerFiles());
            });
            spec.getFrom().attribute(
                            ModAccessTransformExtension.TRANSFORM_ACCESS,
                            false
                    ).attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, ArtifactTypeDefinition.JAR_TYPE)
                    .attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.class, DocsType.SOURCES));
            spec.getTo().attribute(
                            ModAccessTransformExtension.TRANSFORM_ACCESS,
                            true
                    ).attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, ArtifactTypeDefinition.JAR_TYPE)
                    .attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.class, DocsType.SOURCES));
        }
);
v
Even if it works that way, it additionally depends on how the sources are retrieved. If someone just downloads the
sources
jar manually, or uses Gradle to get the Jar with
sources
classifier, or uses an
ArtifactResolutionQuery
to get sources, the transform will not be triggered. Only if some interested party uses proper variant-aware resolution to download the sources, the artifact transform would be used, and of course also then only if it would request your additional transformed attribute, why should it run the transform otherwise? So before you put any more effort into this, the question is, for whom do you want to do this sources transformation. If you do request the sources yourself somewhere and there want to retrieve the transformed source it might work. For all other cases you probably cannot make it work I guess.
1
y
i want to transform the source requested by intellij idea, or adding a configuration and forcing a source download
v
Well, you have to check whether IJ uses proper source variant or not. I think I have seen it two sometimes either, sometimes the other. If it does, you might be able to change the configuration it adds and add your attribute iff it uses a named configuration and not a detached configuration. For your own source downloads you should be able to make it working if done properly.
y
ij get souce by this, i think download by myself solution would be better,but there are some example or document can help me with this?
j
In my experience, artifact transforms simply do not work with sources variants. https://github.com/gradle/gradle/issues/19207
🐼 1
v
That issue is exactly the restriction I said. It is about that consumers of course have to actually use the source variant, and you have to somehow be able to make it request your transformed attribute. If this is not given, no transform can be triggered.
j
Ahh, so I guess since IntelliJ and Eclipse have their own method for grabbing sources, it bypasses Gradle's transformations since it isn't using it.
That makes sense. For years I just kind of assumed it didn't work 😅
v
The "problem" with the snippet @yjxf vf linked to last is, that he just requests the normal sources variant. If you do it properly it works fine. For example this works just fine:
Copy code
abstract class MyTransform : TransformAction<TransformParameters.None> {
    @get:InputArtifact
    abstract val inputArtifact: Provider<FileSystemLocation>

    override fun transform(outputs: TransformOutputs) {
        outputs.file("${inputArtifact.get().asFile.name}.txt").writeText("")
    }
}
dependencies {
    registerTransform(MyTransform::class) {
        from
            .attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.SOURCES))
            .attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, ArtifactTypeDefinition.JAR_TYPE)
        to
            .attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.SOURCES))
            .attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, "transformed-jar")
    }
}
dependencies {
    implementation("io.github.typesafegithub:github-workflows-kt:3.3.0")
}
val foo by tasks.registering {
    doLast {
        configurations.compileClasspath.get().incoming.artifactView {
            withVariantReselection()
            isLenient = true
            attributes {
                attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage.JAVA_RUNTIME))
                attribute(Category.CATEGORY_ATTRIBUTE, objects.named(Category.DOCUMENTATION))
                attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling.EXTERNAL))
                attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.SOURCES))
                attribute(ArtifactTypeDefinition.ARTIFACT_TYPE_ATTRIBUTE, "transformed-jar")
            }
        }.files.forEach { println("FOO: $it") }
    }
}
There are a few pitfalls. If you declare a configuration with the attributes, you do not get the sources of the transitive dependencies, because the source variants to not have dependencies declared typically. If you transform for example the
DOCS_TYPE_ATTRIBUTE
from
sources
to
transformed-sources
, this will not work, because with the artifact transform you have to do
withVariantReselection
and during variant selection artifact transforms are not considered, so it couldn't find a matching variant. With an own attribute, it is the same as with the last case, you would have to make it an attribute of the artifact, not the configuration (like
ARTIFACT_TYPE_ATTRIBUTE
), as then the variant reselection can find the proper variant that can then be transformed with an artifact transform so that the artifact attributes are matching.
j
Ahh ok, that's helpful
v
What we might need generally is an artifact view on an artifact view, so that we can do variant reselection first and then an artifact transform, but that unfortunately does not work afaik
Ahh, so I guess since IntelliJ and Eclipse have their own method for grabbing sources, it bypasses Gradle's transformations since it isn't using it.
As I wrote, I have seen IJ doing both with Gradle projects, using artifact resolution queries or also using source variants. But even if it uses the source variants, it has to do it in a way that you can influence from the build script, so that you can adjust the attributes it uses. If it for example uses a detached configuration you cannot lay your finger on it.
👍 2
j
Ok, then I need to take a closer look at how variant selection and artifact transforms actually work, because it seems like there are still some things I am misunderstanding about them. But from what I understand, the main pitfalls with sources is that a) source variants do not declare dependencies and b) I need to use the ArtifactTypeDefinition attribute since merely selecting the sources docs type attribute doesn't do anything.
v
Ok, then I need to take a closer look at how variant selection and artifact transforms actually work, because it seems like there are still some things I am misunderstanding about them.
Yes, they are not easy to grasp. I also needed quite some time and misunderstandings until I was able to half-way wrap my head around them. And I'm still not always sure I got it right. 😄 That there are different levels of artifacts (on configurations and on artifacts), but you request them the same for example. Or that during variant selection transforms do not come into play yet but you first have to find a matching variant of which artifacts you can then transform the attributes (also the configuration-level ones). That latter just here would not work as you need the original one for the variant selection. ...
a) source variants do not declare dependencies
Not really a pitfall, just something you have to keep in mind. For example if you create a configuration that extends another one and on the configuration set the attributes to get the sources, then you will most likely just get the sources of the directly declared dependencies but not for the transitivies. But that might even be what you want.
b) I need to use the ArtifactTypeDefinition attribute since merely selecting the sources docs type attribute doesn't do anything.
No, not if we are talking about just retrieving them. In my example it was just used so that the "variant reselection" could be done with the original artifacts and then the artrifact-level attribute was used to trigger the transformation. Just to retrieve the sources of all dependencies (also transitive), that have a proper source variant published, this works just fine:
Copy code
ependencies {
    implementation("io.github.typesafegithub:github-workflows-kt:3.3.0")
}
val foo by tasks.registering {
    doLast {
        configurations.compileClasspath.get().incoming.artifactView {
            withVariantReselection()
            isLenient = true
            attributes {
                attribute(Usage.USAGE_ATTRIBUTE, objects.named(Usage.JAVA_RUNTIME))
                attribute(Category.CATEGORY_ATTRIBUTE, objects.named(Category.DOCUMENTATION))
                attribute(Bundling.BUNDLING_ATTRIBUTE, objects.named(Bundling.EXTERNAL))
                attribute(DocsType.DOCS_TYPE_ATTRIBUTE, objects.named(DocsType.SOURCES))
            }
        }.files.forEach { println("FOO: $it") }
    }
}
👍 1
Actually it seems to even work for dependencies that do not have a GMM at all but just a POM. Seems Gradle auto-creates the sources variant for those. Just tried with
commons-io:commons-io:2.19.0
and it worked.
j
Yeah, that much I do know. I think there are some defaults for artifacts that have standard classifiers (i.e.
sources
and
javadoc
).
v
For
"com.fasterxml.jackson.core:jackson-core:2.19.0"
for example it does not work, because they do publish GMM, but do not publish a proper sources variant.
😬 1
Well, they publish Gradle Module Metadata and do not provide that variant, so Gradle adheres to that. Only if something does not have Gradle Module Metadata it auto-creates some variants from the typical setup. I didn't know the sources variant is part of it, or that is new.
If you for example tell the repository to use POMs and ignore Gradle Module Metadata, the
jackson-core
sources can again be retrieved, because then the sources variant is again auto-created from the pom information.
But this really is a publishing bug in
jackson-core
.
This usually happens when projects follow the bad practice to add `artifact(...)`s directly to a publication, instead of adding a proper variant to the
java
component. Especially before
java { withSourcesJar() }
existed this was often made and some did not fix it when that came out.
Here you can read which variants are derived from a POM without GMM: https://docs.gradle.org/current/userguide/variant_attributes.html#sec:maven-mapping-to-variants
Ah, at least in the docs the sources and javadoc derived variants are only mentioned since 7.5 and I've read that chapter before .
Yeah, was also added in 7.5