so I thought I managed to write "my own" custom dependency setup to support python dependencies. But now I'm facing the problem that I need to use a set of those dependencies as a task input and when resolving them I don't get them if I invoke

resolvableConfiguration.incoming.resolutionResult.rootComponent.get().dependencies

Reading around it seems that maybe those dependencies don't match the variant attributes associated to the configuration, but I don't know how I can tweak this so that these self-resolving dependencies "declare" the correct variant. Some code to make this more concrete:

// plugin `apply()`
val declarable = project.configurations.dependencyScope("pythonScope")

project.configurations.consumable("pythonConsumable") {
    it.attributes {
        it.attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class.java, "python"))
    }
}

val resolvable = project.configurations.resolvable("pythonResolvable") {
    it.extendsFrom(declarable.get())

    it.attributes {
        it.attribute(Usage.USAGE_ATTRIBUTE, project.objects.named(Usage::class.java, "python"))
    }
}

// custom dependency class
interface PipDependency : Dependency {
    fun asPipRequirement(): String
}

data class RemotePipDependency(
    private val name: String,
    private val versionSpec: VersionConstraint,
    private val extras: List<String>,
    private val objectFactory: ObjectFactory,
) : PipDependency, SelfResolvingDependencyInternal, FileCollectionDependency {
    // ... etc etc, not much going on here
}

// plugin extension to encapsulate the custom dependency
internal class DefaultPipDependencyExtension @Inject constructor(
    private val dependencies: DependencySet,
    private val objectFactory: ObjectFactory,
) : PipDependencyExtension {
    override fun invoke(name: String, versionSpec: String?, extras: List<String>): Dependency =
        RemotePipDependency(
            name,
            versionSpec?.let { VersionConstraint(it) } ?: VersionConstraint.EMPTY,
            extras,
            objectFactory,
        ).also { dependencies.add(it) }

    override fun invoke(project: ProjectDependency): Dependency = project.also { dependencies.add(it) }
}

If from a task I call

println(resolvable.get().incoming.resolutionResult.rootComponent.get().dependencies)

I get nothing.

Maybe because you did not declare any dependency? 😄

ah eh sorry, I forgot to post that part

Besides that you of course should neither use internal classes, nor self-resolving dependency which are deprecated 🙂

sorry for the ignorance, but what is a MCVE?

you were the one suggesting me using self-resolving dependencies in the first place 😅

But were we then not told that this is bad by someone?

but, aside from the bantering, it's also what Kotlin JS uses (and why the "internal" version exists afaict)

🤷‍♂️😄

I think it's super cool that you're trying to get Python working better in Gradle. Unfortunately Gradle sucks for non-JVM dependencies, there's zero support :( Some plugins (see below) hack-around this by spinning up a localhost server that pretends to be a Maven or Ivy repo, and internal converts the request into an external repo request. It could be a lot of work but unfortunately it's the only Gradle compatible approach. • https://github.com/JetBrains/intellij-platform-gradle-plugin/blob/v2.0.1/src/main/kotlin/org/jetbrains/intellij/platform/gradle/shim/PluginArtifactoryShim.kt#L21 • https://github.com/gradlets/gradle-typescript/blob/1.4.1/gradle-typescript/src/main/java/com/gradlets/gradle/typescript/shim/NpmArtifactoryShim.java#L45

it's a very not ideal situation. For half of what I need tapping into the configuration mechanism seems too much, and yet I need to in order to have the artifact of the plugin usable between subprojects.

I think I also suggested the proxy approach @Adam, he just didn't like it too much and preferred to use the internal classes the Kotlin/JS plugin is using 🙂

thanks @Adam that's what @Vampire suggested in the first place too. Truth is that I don't want Gradle to fully take over dependency resolution and I still need to put the dependency declarations in a pyproject.toml file, but if that's the only way I can get this done then I'll have to bite this bullet I guess

I still need to put the dependency declarations in a pyproject.toml file

Do you mean that the dependencies will be declared in two places, the Python toml and Gradle

dependencies {}

?

no, the goal would be to generate the pyproject.toml file from the plugin using the declared dependencies. For this technically I don't need to have the dependencies resolved though

Ah gotcha

yes that could work for remote dependencies, but the other half of the problem is that I want to be able to do something like

dependencies {
  pip(project(grpc.stub.python))
}

and for this it's definitely easier to tap into the variant-aware resolution system

true, but you could still do that under the hood with a custom

pythonDependencies {}

block. It would be even easier because the standard

dependencies {}

block is quite old, and is very Groovy based. I'd create a

PythonDependencies

class and add methods with overloads. •

fun pip(coord: String)

would be put straight into the

pyproject.toml

•

fun pip(project: Project)

would use the Gradle dependency resolution.

and to be fair, the plugin in its current state sort of works already, but I'm trying to make the set of dependencies an output of the task that generates the toml file to get the full advantage of Gradle caching and make the code slightly leaner and more readable. But in order to do that, according to the docs at least, I need to use the resolvable configuration

yeah, exactly

but to be honest, I'm still also curious to understand why my attempt failed. Is it because self-resolving dependencies don't and can't provide metadata and Gradle cannot figure out which variant they belong to?

I need to use the resolvable configuration

Yeah, for sharing files between subprojects you need a 'declarable' configuration and a 'resolver' and a 'consumable'. It's a lot of boilerplate. Just to be clear, in case there's some confusion, the docs you linked are about getting the dependency graph. To get the actual files it's simpler:

val fooConfResolver by configurations.creating {
  extendsFrom(fooConf)
  isCanBeDeclared = false
  isCanBeConsumed = false
  isCanBeResolved = true
}

val fetchFiles by tasks.registering(Sync::class) {
  from(fooConfResolver.incoming.files)
  into("fetched-files")
}

yeah the thing I don't need the files there, I need the name and version specifications I can find in the

declarable

. I'm also not sure if it's resolved too early, it's inside the

doLast

of the task that otherwise reads from the

declarable

configuration

was it registered as a task input correctly? (Or you're running with

--no-build-cache

&

--no-configuration-cache

)?

umm, no I don't think so and definitely not. I was just "testing out" the approach suggested by the documentation

okay cool, well it sounds like you've got the basics done correctly

yup, before trying this I was just passing

declared.get().dependencies

as an input in a property. This works but breaks the build the moment I try to define an output for the task, I guess because something in the tree of classes making up the set of

RemotePipDependency

is not serializable and thus Gradle cannot cache it. The sample you pointed at is precisely where I got the line about

incoming.resolutionResult.rootComponent

.

ahh okay - and was there a task in the sample for logging the declared dependencies? I remember seeing that somewhere...

That’s literally me for the last 3 weeks 😂