Hi, what is a correct way to share source code subsets between modules in multi-module project? For example, I have project with lib-A -> lib-B -> [service-1, services-2] with a specific subset of source files in each of them. I have custom plugin that generates stuff out of aggregated sources files at service-1 and services-2. Is it right way to go by exposing sources as custom configuration artifacts? How to do that correct? I have tried different approaches, but none of them worked as expected.

The "correct" approach if I got your right is not to do it. Having the same source means for produce the same classes and this have the same classes in multiple jars which you shouldn't so, besides that you also waste time by compiling it multiple times. If you have code you need in more than one module, the correct approach is to have this code in another module on which the others both depend.

Lets say it's not a java code in that libs. Actually it is, but it's gwt. The compiler needs source files at each service and yes, I am aware of that you have said about double compilation. But I have not so standard case with gwt modules) Our current implementation makes even dirtier hacks, I am trying to do it more gradle way in steps, as the first step I want to expose gwt sources as project artifacts to other subprojects via configurations. Later maybe I will try to expose gwt compiler output at each module, but right now it seems to be quite a step

To just share all the sources, it should be sufficient to have

java { withSourcesJar() }

, as that not only crates the task for creating the is jar, but also configures an outgoing variant for it and configures the component for publishing, so you can just depend on that variant by attributes it use an artifact view. But exposing the GWT compiler output as outgoing variant should also be relatively simple. The hardest part usually is to come up with the best values for the attributes. The rest is just creating a consumable configuration, setting the attributes, and adding the artifact(s). If you want to only share a subset of the sources, the effort is probably exactly the same. This is the according docs: https://docs.gradle.org/current/userguide/how_to_share_outputs_between_projects.html#variant-aware-sharing

I have made everything work with sourceSets and configurations, but I cant figure out how to fix transitivity. Here is my configurations:

val gwtSourcesConfiguration = project.configurations.consumable(GWT_SOURCES_CONFIGURATION) {
    attributes {
       attribute(Category.CATEGORY_ATTRIBUTE, project.objects.named(Category.LIBRARY))
       attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, project.objects.named(GWT_SOURCES_JAR))
    }
}

val gwtImplementation = project.configurations.dependencyScope(GWT_IMPLEMENTATION) {
    extendsFrom(gwtSourcesConfiguration.get())
}

project.configurations.dependencyScope(GWT_API) {
    isTransitive = true
    extendsFrom(
       gwtImplementation.get()
    )
}
project.configurations.resolvable(GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION) {
    description = "Dependencies for GWT compilation"
    extendsFrom(
       project.configurations[GWT_API],
       project.configurations[GWT_IMPLEMENTATION],
    )
    attributes {
       attribute(LibraryElements.LIBRARY_ELEMENTS_ATTRIBUTE, project.objects.named(GWT_SOURCES_JAR))
    }
}

When I'm calling resolve on GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION, it doesn't provide transitive dependencies, despite projects added as api dependency

dependencies.add(GWT_IMPLEMENTATION, project(":libs:server")) //at service gradle

dependencies.add(GWT_API, project(":libs:gwtlib")) // at  ":libs:server"

I am receiving libsserver but not libsgwtlib Can anyone figure out the problem?

Besides that it looks suspicious that a dependency scope configuration extends a consumable configuration, ... ..., the problem most probably is, that this variant does not bear the dependency information you expect to get. Maybe you should instead us an artifact view with variant reselection to get the sources. Because then the dependency tree is first resolved with the normal variants which have the dependency information, and then with the artifact view you change for each the variant to the sources one and thus should also have the transitive sources.

that it looks suspicious that a dependency scope configuration extends a consumable configuration,

yeah, I have added this to check if that helps) I will remove that

If

GWT_SOURCES_CONFIGURATION

would extend

apiElements

, so that it inherits the dependencies, that could alternatively maybe also work as then the variant would have the dependencies you are after I think.

Hm, maybe I can add variant to jvm api/implementation somehow? then I get rid from GWT_IMPLEMENTATION and GWT_API and project dependencies duplication because for now I have something like this:

implementation(project(":libs:server"))
gwtApi(project(":libs:server"))

The only thing - how to ensure that gwt sources jars will be only available for

GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION

?

Hm, maybe I can add variant to jvm api/implementation somehow?

api

and

implementation

are configurations, you add variants to a project.

then I get rid from GWT_IMPLEMENTATION and GWT_API and project dependencies duplication

because for now I have something like this:

You can also make GWT_IMPLEMENTATION extend implementation and GWT_API extend api, then all dependencies that are in implementation are automatically also in GWT_IMPLEMENTATION and all dependencies that are in api are automatically also in GWT_API. But that only makes sense if you want to add additional dependencies to GWT_*, otherwise it is just clutter and you can just use api and implementation directly.

I have tried apiElements, but now I receive all jars, not just that I needed

Hard to guess from just that sentence what you did exactly, what you expected, and what you got. Maybe you can share an MCVE to make it clearer.

The only thing - how to ensure that gwt sources jars will be only available for

If you request the attributes you want from a configuration or dependency or through an artifact view, then you only get those variants that match those attributes. Unless you add the necessary dependencies (directly or through

extendsFrom

) to the

GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION

on producer side, you would only get the directly declared dependencies when requesting the according attribute on the consumer side through dependency or configuration, as during dependency graph calculation you would only get those variants which do not have the necessary dependencies. But if you as I said request on consumer side for example in the configuration, the normal attributes, you get the full dependency tree with transitive dependencies, and then through an artifact view where you request the attributes for the source jars with variant reselection, you would get from all those dependencies the source variant, including the transitive ones.

If I understand you correct, if I have resolvable and consumable configurations, which shares matching attributes, and do not set any extendsFrom, projects decalared with implementation/api dependencies without configuring them additionally, then resolvable configuration will get artifacts from consumable by attributes? like this:

project.configurations.consumable(GWT_SOURCES_CONFIGURATION) {
			attributes {
				attribute(Attribute.of("sourcesSubset", String::class.java), "gwt")
			}
		}

project.configurations.resolvable(GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION) {
			description = "Dependencies for GWT compilation"
			attributes {
				attribute(Attribute.of("sourcesSubset", String::class.java), "gwt")			
			}
		}

I am adding artifacts like this:

project.afterEvaluate {
			jar.configure {
				from(gwtSources)
			}
			project.artifacts.add(GWT_SOURCES_CONFIGURATION, jar)
		}

project declared as api and implementation now, if I got you correct that should be enough though at leaf project I don't get any dependencies

val resolved = project.configurations[GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION].resolve()
		resolved.forEach { println("RESOLVED GWT_COMPILE: ${it.absolutePath}") }

No, again, you request the attribute in

GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION

which means it is already taken into account when building the dependency tree and so you miss the dependencies that are not declared for those variants currently. With that setup you would request the normal attributes and then use an artifact view with variant reselection to get the source jars.

Fwiw, I had made a sample with GWT (note: directly using source directories, not packaging them in a JAR): https://github.com/tbroyer/gwt-gradle-example This approach is used in a project at work for a few years, with 40 or so subprojects, 10 of which are used in a GWT application, and only 7 of those being direct dependencies; so transitives do work! (the sample should have one transitive too, as a proof that it works) Feel free to use it as an inspiration. Disclaimer: I haven't read the whole conversation, only the message cross-posted to the channel.

I have tried ArtifactsView,

val resolved = project.configurations.getByName(GWT_COMPILE_SOURCES_DEPENDENCIES_CONFIGURATION)
    .incoming
    .artifactView {
       withVariantReselection()
       attributes {
          attribute(gwtSourcesArtifactAttribute, GWT_SOURCES_ATTRIBUTE_VALUE)
       }
    }
    .files
resolved.forEach { println("RESOLVED GWT_COMPILE: ${it.absolutePath}") }

Still get all jars, no just with declared with that attribute

Still get all jars, no just with declared with that attribute

So again, an MCVE might be helpful 🙂

@Vampire https://github.com/Guchman/gradle-configurations-mcve

./gradlew checkGwtSources

checks invariants of :service configurations as I understand them and expected to pass though it's not :( Maybe I understood you wrong somehow

Besides that there are maaany bad practices in there, your main Problem is, that you did not wrap your head around the attribute matching algorithm yet. And also when inventing own attributes you have to add them to the attribute schema. Regarding the matching, if you request a value for an attribute and the producer does not provide that attribute at all, this is considered compatible. So by only having that one custom attribute, all variants are compatible, I actually wonder that you didn't get ambiguous problems. Also on your resolvable configuration you did not request any attributes at all, which is also not a "great" idea. But if you for example use the standard attributes that Gradle also uses for the sources variant and use for the docs type

gwtSources

instead of

sources

, your checks start to become green.

Can you please share what bad practices you are talking about? And can you maybe recommend some source of knowledge on gradle good/bad practices (except the gradle docs)? I am working with gradle a lot and I really want to fix the gaps

For example: • Never user

allprojects { ... }

,

subprojects { ... }

, or any other form of cross-project configuration, for example for the repositories use

dependencyResolutionManagement

in the settings script, for other common build logic, use convention plugins. • Don't use

apply()

but use the

plugins { ... }

block to apply plugins. • Don't use

-all

distribution, except for 2 edge case using it is just a waste of time, disk space, and bandwidth for everyone and everything executing the build. • Don't use

confiugre<JavaPluginExtension> { ... }

and similar, but

java { ... }

... • Don't use

project

at task execution time • Don't call

resolve

on a

Configuration

manually • ...

Oh, thanks, I will pay more attention on this stuff at projects, though convention plugin is a bit overkill for exampler project :)

Well, you do have convention plugins in there already. But yeah, if it was just for the MCVE, that's fine of course. :-)