I updated an android build script from groovy to k...
# community-support
c
I updated an android build script from groovy to kts. but now I have my github action failing because there's a bunch of ENV variables that can't be found for certain steps in my build process. The fix was easy. Add my env variables to the different steps that "need" them. But they never needed them before. The "task" that's being run doesn't use those env variables. My assumption is that whenever you run a gradle task from a kts file it "validates"/compiles the entire build file, unlike groovy. And because of that fact, and the fact that there are other sections of the kts file that do require an ENV variable, it now requires me to give that step in github actions access to those env variables. Does that seem like the correct assumption on why groovy vs kts is different?
t
How do you access those variables in your script?
c
~System.getEnv()
Which has worked fine with the build.gradle file. but once we moved to build.gradle.kts it seems to want to validate the entire script before it just runs a tiny little task (thats unrelated to gradle in this case) for example. our task is called "getReleaseNotes" and it just runs a git command. and its funny that now "getReleaseNotes" is broken because it can't find env variables that are delcared elsewhere in the script.
so. short term. id like to just stop requiring the env variables for our "getReleaseNotes" step. long term. that should be a bash script and not a gradle task. lol
t
Maybe if you show snippets of before/after code? Because I see no reason why this would happen unless you made a mistake in your build script.
c
Basically running this gradle task fails now (in kts) even though it worked in groovy
Copy code
tasks.register("getCommitMessages") {
             doLast {
                 println("print release notes")
            ...
             }
}
it fails and complains that
ANDROID_KEYSTORE_FILEPATH
env var is missing. But getCommitMessages doesn't ever call any env vars. But elsewhere in the script (in the android block) we request
ANDROID_KEYSTORE_FILEPATH
in the signingConfigs block. So I'm just trying to sanity check why kts is resolving/compiling the whole file.
t
What I was asking was about the parts that do use envvars (that said, I don't do Android so might not be able to help if it's in Android-specific parts)
c
Copy code
signingConfigs {

         create("release") {
             storeFile = file(System.getenv(["ANDROID_KEYSTORE_FILEPATH"])
             storePassword = "password"
             keyAlias = "staging"
             keyPassword = "password"
         }
     }
That signing configs block exists inside of the android block. while the tasks.register exists inside of the buildTypes block, which is inside of the android block.
c
https://docs.gradle.org/current/userguide/build_environment.html#sec:gradle_environment_variables - see the
lazy
reference to defer accessing environment variables until they are needed (if at all).
👀 1
c
Am I missing something? I don't see "lazy" on that docs page?
t
Have you tried using
register("release")
rather than
create("release")
(again, I don't know the AGP DSL, so it might not even exist) Generally speaking, using
create()
means the associated configuration block is executed before the build runs (think of it as executing immediately), whereas using
register()
defers execution of that block to when it's known to be needed. See https://docs.gradle.org/current/userguide/task_configuration_avoidance.html and https://docs.gradle.org/current/userguide/lazy_configuration.html
c
this:
Gradle provides a number of environment variables, which are listed below. Environment variables can be retrieved lazily using
providers.environmentVariable()
.
the problem is that env vars, when accessed via
System.getenv
, happens at configuration time across all the tasks (not just the ones that will be run) - switching them to a provider defers resolving the env var (and makes things compatible with the configuration cache).
c
oooh. very interesting (and damn cmd+f failing me)
but i guess thats a difference between groovy and kts right? thats moreso what im questiniong. trying to figure out "why" this broke when I "simply" changed the file extension to kts pretty much (there wasn't too many things I had to change since its a basic project)
oh. System.getenv makes it incompatible with config cache? I guess I really want to move to provides.envrionementVariable then!
c
that lifecycle is the same between Groovy and Kotlin DSLs; something else got adjusted in translation to make the resolution eager. Its often hard to tell from Groovy what actual API call happens on the backend, so translating one-to-one can be challenging.
👆 1
Plugins and build scripts may read system properties and environment variables directly at configuration time with standard Java, Groovy, or Kotlin APIs or with the value supplier APIs. Doing so makes such variable or property a build configuration input, so changing the value invalidates the configuration cache. The configuration cache report includes a list of these build configuration inputs to help track them.
In general, you should avoid reading the value of system properties and environment variables at configuration time, to avoid cache misses when value changes. Instead, you can connect the `Provider`returned by providers.systemProperty() or providers.environmentVariable() to task properties.
https://docs.gradle.org/current/userguide/configuration_cache.html
not really “incompatible” with configuration cache (there are many other patterns that are hard incompatibilities), but less than ideal.
c
something else got adjusted in translation to make the resolution eager. Its often hard to tell from Groovy what actual API call happens on the backend, so translating one-to-one can be challenging.
interesting. i will take another look, but thats actually exactly what i was looking for. that it was always eager and its not just a groovy vs kts thing
for a second i thought maybe kts precompiles things or something since its "static" vs dynamic "groovy" but also. im still a gradle noob and still learning. hence why i was just trying to make sense of this.
c
all good. note there are a few different points in the lifecycle: • script execution time - when the script is run, at the start of configuration - this generally registers a bunch of configuration callbacks to be called later. This is where the opaqueness of Groovy make it challenging to translate - perhaps something that was a callback isn’t now, etc; e,g. dropping a
println
at the root level of script happens here (at inner levels it would be subject to when callbacks get invoked) • configuration time - really the results of executing the script, as noted above - what is needed to be configured is executed; • execution time - the necessary tasks are executed
e
System.getenv
returns a nullable string, but it's possible that Kotlin is enforcing nullability checking on APIs where Groovy does not
if the APIs you're using take providers, then
providers.environmentVariable
is lazy
so looking into it, Android signing might not be lazily configurable, but you can manually
Copy code
signingConfigs {
    create("release") {
        val storeFilePath = System.getenv("ANDROID_KEYSTORE_FILEPATH")
        if (storeFilePath.isNullOrEmpty()) return@create
        storeFile = file(storeFilePath)
        storePassword = "password"
        keyAlias = "staging"
        keyPassword = "password"
    }
}
for tasks that don't use release signing, it won't matter if it's unconfigured
c
System.getenv
returns a nullable string, but it's possible that Kotlin is enforcing nullability checking on APIs where Groovy does not
oooh. interesting. so i can't actually use
providers.environmentVariable
in android signing?
e
doesn't look like you can
😍 1
c
such a bummer. CI issues are preventing me from adding the env vars to all of these steps. back to square 1. hm....
v
for a second i thought maybe kts precompiles things or something since its "static" vs dynamic "groovy"
That's actually irrelevant. Yes, Kotlin code needs to be compiled of course. I think - but am not sure right now - the Groovy code is also compiled, so that it does not need to be interpreted each time it is run if it did not change. But whether you compile it or not before you run it is irrelevant.
System.getenv(...)
will never be executed at compilation time, that would be extremely strange, wrong, strange, lead to wrong results, and strange. If there is a difference, then as the others said maybe because in Groovy something was lazy which it is no longer in Kotlin, while it usually is the other way around due to backwards compatibility. For example if you have
Copy code
tasks.foo {
   ...
}
this is eager in Groovy but lazy in Kotlin. I'm not right now aware of a case where it is the other way around. But especially with Android everything is possible, as Android is always a bit special. It's really hard to tell what the difference might be without having both versions and looking hands-on.
oooh. interesting. so i can't actually use
providers.environmentVariable
in android signing?
Sure you can, just does not make much sense if you need the configuration at configuration time because the property you configure is not a lazy Gradle property. If what you configure there is a https://developer.android.com/reference/tools/gradle-api/8.3/com/android/build/api/dsl/SigningConfig, then the
storeFile
property is a plain
File
so whether you use
System.env
or
providers.environmentVariable
makes no difference, you can use both. But you need the value at configuration time and it will be a configuration cache input if you use CC.
That signing configs block exists inside of the android block. while the tasks.register exists inside of the buildTypes block, which is inside of the android block.
The block you showed will not even compile in Kotlin. In Groovy it will compile but fail at runtime as
ArrayList
cannot be cast to
String
and parentheses are not balanced. The call itself will either way not be the cause of the difference, but the context in which it happens, but it shows that any advice that can be given based on the examples you show are just wild guesses as you did not show the actual code. 😉 Maybe before you had the
release
signing config lazily registered and now by doing the
create
call it is immediately realized and configured instead of when needed first like mentioned already. Or you changed something else that causes now that part of the configuration to be executed which can be a totally different and unrelated part. For example if you now somewhere cause all tasks to be realized and one of the configured task uses that configuration and realizes it when in the old version the task was not configured and so that
System.getenv
call never been done, ... But as I said, without the full and correct code hands-on it is hard to guess where your behavior change is coming from.
❤️ 1
c
Alright, apologies for the delay but needed to get my EM's thumbs up to include essentially our entire build file (even though its pretty standard) https://gist.github.com/ColtonIdle/e2383a827db1b57334f135837f47b020 Anyway. so basically moving from groovy to kts and our CI complained because it seems like its eagerly grabbing env vars now. not sure if that helps with the understanding of my issue. but yeah. thanks in advance if anything else jumps out at ya.
e
so it's exactly what I said earlier,
System.getenv
returns a nullable string, but it's possible that Kotlin is enforcing nullability checking on APIs where Groovy does not
file()
takes a non-null argument
in Groovy you're stringifying, and passing in
"null"
if the environment variable doesn't exist. in Kotlin you aren't
you could write
Copy code
storeFile = file("${System.getenv("KEYSTORE_FILE")}")
or
Copy code
storeFile = file(System.getenv("KEYSTORE_FILE").toString())
which would then have the same behavior as your Groovy script, or just don't set it at all if not present,
Copy code
System.getenv("KEYSTORE_FILE")?.let { storeFile = file(it) }
or
Copy code
storeFile = System.getenv("KEYSTORE_FILE")?.let(::file)
❤️ 1
v
And, your build is broken, even in the Groovy version. You just might have been lucky that you didn''t hit the breakage. You use
File("release.txt")
(resp.
new File("release.txt")
). Using the
File
constructor (and similar things like
FileInputStream
) with a relative path resolves to the current working directory. This often is the root project directory but it is not guaranteed and also often not the case, but the IDE installation directory, or the daemon log directory, ... Almost any JVM code, not only Gradle build logic, that depends on the current working directory is broken, because if the user calls the program from a different directory it will not work as expected. The only use-case I'm aware where this is appropriate is, to resolve a relative path given by the user on the commandline as you can then safely assume he means relative to the current working directory and in the Gradle case even that would not be right, as the working directory of the daemon has nothing to do with the working direcotry of the user, even though it often is the same.
❤️ 1
c
thank you all. will re-read this when im at my work machine, but it does seem kinda funny to me how storeFile = file("${System.getenv("KEYSTORE_FILE")}") could be the solution lol. just wrapping it in a string 😄
e
that's what your groovy code was doing in the first place. which it shouldn't have been, but 🤷
☝️ 2
😱 1
c
Sorry. What do you mean "it shouldn't have been"? Is there a better way to do this? Unfortunately I also tried to abstract this "selection" of picking an env variable/local property into a function like
Copy code
fun getProperty(propertyName: String) : String {
  val prop = project.properties[propertyName]
    ?: gradleLocalProperties(rootDir, providers).getProperty(propertyName)
    ?: System.getenv()[propertyName]
    ?: throw IllegalStateException("Add a local property or env variable")

return prop
}
but that seemed to cause even more issues. 🙃
c
file("${System.getenv("KEYSTORE_FILE")}")
==
file("null")
. Obviously incorrect, saved by the fact that those tasks aren’t run and try to access that file. One could use the lazy properties (where available) or set the value to null (storeFile = null) where the env var doesn’t exist.
t
IOW, you could have written
file(System.env.KEYSTORE_FILE)
in Groovy, and you'd have had the same problem as in Kotlin. So either you translate to Kotlin with the same behavior (wrap in a string), or you fix it using the suggestions from above (
System.get("KEYSTORE_FILE")?.let(::file)
)
☝️ 2
c
Thanks everyone for teaching. This has been enlightening!
👌 1