Looking for ideas/thoughts on the following Groovy...
# plugin-development
j
Looking for ideas/thoughts on the following Groovy DSL / Kotlin DSL topic. Context We are developing a Gradle-based build system for embedded development. It is a custom build system that only uses the Gradle core systems (e.g. no Java etc). The system will replace an existing build system that is completely self-made. There are many huge projects that will be migrated from that old system to the Gradle-based system. The old system offered a lot of customization options that are in use. The main motivation for moving to Gradle are the performance characteristics of Gradle's core systems: incremental building, parallelization of tasks, build caching, dependency resolution and dependency caching (that we'll use for custom tool provisioning). On the long run, the new system should also have a stronger "convention-over-configuration" approach. The old system requires a lot of individual configuration. With the move to Gradle, we have the option to design our own build configuration DSL that is more compact than what the old system has and can offer more conventions/defaults. Before we get there, the existing projects need to be migrated. Which we plan to do in some semi-automatic way. This will take a lot of configuration data over into the Gradle scripts that has to go somewhere. Question/Problem Due to the above, each project will have quite some individual build configuration. At least initially (how much it can be reduced later is unclear at this point). If we put all of this configuration in one
build.gradle(.kts)
file, we end up with a file of several 100 lines for some projects. Which is not practical. Therefore, we want to split up the configuration. In our build system, we have concept called "*process*" which corresponds to one or more Gradle tasks. A project can have several 100 of these processes. We want to have one configuration file per process. How do we achieve this best? Here is what we tried/do now: • Using pre-compiled script plugins is not an option. The files are something developers work with regularly. Changing a configuration cannot invalidate the whole build. Which pre-compiled script plugins do, as changing them changes the plugin classpath. • We started using Kotlin DSL for good IDE support in IntelliJ for our custom DSL/extension. • But for the split-up, the only option we found is "traditional" script plugins in Groovy DSL. Now we have a mix of
build.gradle.kts
(Kotlin) and
.gradle
(Groovy) script files. ◦ (Is there a way to use the Kotlin DSL here with a custom extension? I think there isn't right?) • Many developers will probably use VS Code as they use it for the actual development as well. And the Kotlin DSL support is not good here, which weakens the argument for Kotlin DSL in general. I am not happy with the mixed DSL situation we have right now. The only "acceptable" solution I see right now is moving completely to Groovy DSL. But also there the IDE support is bad even in IDEA. Our custom extension is not recognized in the script plugins (
.gradle
files). I am not very satisfied with the solution because of that. And I wonder if there is any other alternative I am missing.
p
(Is there a way to use the Kotlin DSL here with a custom extension? I think there isn't right?)
The Kotlin DSL doesn't generate accessors to extensions in applied scripts because it can't know for sure they will be present.
You can still use the "API" to access the extensions as follows:
Copy code
configure<ExtensionType> {
    myProp = "value"
}
The main issue is that types from buildSrc won't be visible in these scripts in IDEs (only types from
gradleApi()
are). You would need to publish an artifact with the extension type and add the
buildscript {}
dependency to each of these scripts. Not a good solution at all, but that's how it could be possible 😕
Did you consider using external files in other formats?
j
Thanks Paul. This is helpful, haven't thought of it for some reason:
Copy code
configure<ExtensionType> {
    myProp = "value"
}
This may be a good compromise. I'll try and see how well it works in the IDEs. (
buildscript {}
part should not be an issue, we do not use buidSrc, but publish in any case to share our "build system" across all projects. And we have settings plugin to control such things if needed).
Did you consider using external files in other formats?
Yes we thought about it but did not really consider it as an option. At least for now. A "Gradle DSL" allows us to quickly iterate as it is simple to define and change (once you know how things work). It'll take time to get to a first "stable state". Once we are there, we might reconsider. But if the IDE support is okaish with the Gradle DSL, I would like to stick with it. It's a big advantage in my mind.
👍 2
a
@Jendrik Johannes, is this new build system for embedded development something you plan to eventually open-source?
j
@Alex Semin Many things are specific to how development is done within the company. But they are generally open to open sourcing things. We will have to see which parts are generic and which are specific. I hope that we will get to a point where we can do that separation. And then think about open-sourcing something. (will be some months until we are there) One thing I definitely want to move out into a open-source plugin is C compilation (which is one part of the build system). We use the C support Gradle has, because it is actually quite good from the performance characteristics. It's just very hard to use out of the "Gradle core native" context. E.g. only use the task for C compilation with custom compilers is not supported without going into internals. I started to move this out here: (only prototype right now): https://github.com/jjohannes/plain-c I also created this Gradle issue for the topic: https://github.com/gradle/gradle/issues/26880
👍 1
a
Thank you for the details!
j
@Paul Merlin thanks a bunch for your input. I got it working pretty much as I wished for. For my extension(s) I can define the accessory manually as Kotlin extension functions in our plugins and then I can use them event without
configure<>
workaround! 💚 Do you know how well IDEA support for this is/was. I am seeing the following. At first I though that it can't work if for IDEA these are just files without context. But then, if IDEA gets all the "configuration events" from Gradle, it could actually work. 🤔 Do you know if this ever worked?
p
Adding to standalone scripts used to work. It doesn't for you?
j
When I click the button I get this:
Then when I click "Load configuration", the other buttons dissapear but nothing seems to happen.
And nothing is working in terms of code completion or error highlighting.
I tried adding this as well, which I think is what you suggested further up (?)
Copy code
buildscript {
    repositories {
        maven("repo-with-my-plugin-and-my-plugins-dependencies")
    }
    dependencies {
        classpath("my.plugin:coordinates:1.0")
    }
}
But it does not seem to make a difference.
p
Grmbl. Then it seems IJ is simply ignoring these files and just highlights them without further context.Could you please open a Youtrack issue and share it here?
j
👍 sure. Thanks for the context. Maybe there are already issues. I now know better what to look for. Will share the issue here once I have created it.
p
Thank you Jendrik!