https://gradle.com/ logo
Join Slack
Powered by
This message was deleted.
# community-support
s
This message was deleted.
c
possible that those plugins call your version object (
project.version.toString()
) multiple times, incurring latency?
j
I was thinking about that, but why it was working with the previous implementation with no issues 🤔
c
perhaps check whether it’s called multiple times (and what the length of a call looks like) - if it is called multiple times likely want to memoize that, returned cached result after the first call.
j
the first difference it was an string and know it is a custom object built with the value source
c
that means that every version.toString() call will execute the value source.
j
yep, I will try that, thank you 🙂
c
may perhaps be wrong on ValueSource being called multiple times (Providers are) - it looks like it only calls obtain() once, according to the docs.
j
Copy code
private fun Project.configureLazyVersion() {
        version = LazyVersion(VersionValueSource.register(this))

        // Some third party plugin breaks lazy configuration by calling `project.version`
        // too early based on plugins order, applying the calculated version in
        // `afterEvaluate` fix it
        afterEvaluate { it.version = LazyVersion(VersionValueSource.register(this)) }
    }
I am calling it twice, but it shouldn't be a problem
Copy code
public companion object {
        internal fun register(project: Project): Provider<String> =
            with(project) {
                providers
                    .of(VersionValueSource::class) {
                        it.parameters.gitDir.set(layout.dir(provider { gitDir }))
                        it.parameters.tagPrefixProperty.set(tagPrefixProperty)
                        it.parameters.projectTagPrefix.set(semverExtension.tagPrefix)
                        it.parameters.stageProperty.set(stageProperty)
                        it.parameters.scopeProperty.set(scopeProperty)
                        it.parameters.creatingSemverTag.set(isCreatingSemverTag)
                        it.parameters.checkClean.set(checkCleanProperty)
                    }
                    .forUseAtConfigurationTime()
            }
    }
c
can you temporarily replace the LazyVersion.toString() with a constant value to see if that changes the behaviour?
j
yeah, with a constant string it is not happening for sure, the configuration is instant
c
yea. you’ll need to start untangling what inside that toString() is the culprit - is something take a long time, is it perhaps locking, etc.
j
curiously it should be a really similar configuration to the previous one, but just wrapped with the value source
c
the theory is always good until the code gets executed 😉 🤦
j
I will try a few things like memoizing
Just a quick test, this "fix" the issue, so the problem is that provider which comes from the value source. Not sure if assigning the string provider to a variable, and then setting it to the project version should be enough
Copy code
public class LazyVersion(internal val version: Provider<String>) {

    private var cachedVersion: String? = null

    override fun toString(): String {
        if (cachedVersion != null) cachedVersion = version.get()
        return cachedVersion ?: version.get()
    }
}
c
that should do it. the provider isn’t likely the problem, as the previous change to return a constant (going through the same provider) also resolved the issue. Glad this is working for you!
j
Thank you 🙂
Wop, I tested it wrong and the problem persists
c
toString logic not quite right - flip the conditional:
Copy code
if (cachedVersion == null) cachedVersion = version.get()
return cachedVersion ?: version.get()
which allows you to simplify to:
Copy code
if (cachedVersion == null) cachedVersion = version.get()
return cachedVersion
j
Looks like it is too late for my head, my bad! I am gonna try
I have to keep 
?: version.get()
, cachedVersion is still nullable, the only way to avoid it is using let
or assigning it to a 
val
c
or 
!!
as you know it can’t ever be null.
👍 1
j
looks like the issue persists
c
does it persist setting cachedVersion to a constant?
val cachedVersion = "1.2.0"
, as a test, avoiding calling the provider.
j
with that it works, but I have commented the version.get, I am going to put it elsewhere
yeah, looks like the problem is not assigning it to a variable, it just calling it
not sure if I should report this issue or I am doing something wrong when I create it
c
looks like you’ll need to track down what inside of that call is causing the latency. As memoizing didn’t help it looks like the one call is the problem.
j
I am going to return a constant directly from the value source, if it works, then the problem is the calculation
👍 1
I am gonna try to catch the version inside the value source, not sure if that could break tests
looks like trying to cache it in a similar way to the 
LazyVersion
is not enough to fix this issue, maybe the instance is recreated constantly?
c
ah yea, looking above you’re creating multiple LazyVersion instances (where the caching was), may need to move that up a level.
j
I am creating two, but I don't thing that removing one would fix this issue, I am gonna try tho
The calculation assigned directly to the project version is slowing the build too, but I haven't changed it too much, I will check it, thank you for your time 🙂
maybe sharing the state in a build service with the jgit results and using it in all projects can solve this issue
c
that would work. you could also have the plugin detect if it is a subproject and set 
project.version = project.rootProject.version
j
That can have three issues: • Maybe it breaks project isolation • I force the author to apply the plugin to the root project • Currently I am supporting having different version based on git tags, so the root project can have the prefix 
v
and a subproject one 
w
, with different versions.
every time I think about the last one I think I'm masoca tho
Indeed it is likely a combination of both, after caching, the time has been reduced but it was still too high compared to not applying the plugin, and after checking it, the version was called a lot of times, but even being called one time per project, it is still 2.5 seconds
Copy code
TIME: 2509
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
TIME: 0
v
You can also use the Gradle Profiler to profile your code
👀 1
j
I think the problem is that I am creating a jgit instance every time because I couldn't pass to the BuildService and the ValueSource as jgit is not serializable. Not sure if it is a good practice to have it as nullable property in the ValueSource and after instantiating it in the buildService, pass it to the ValueSource. Not a good Kotlin code, but I am not sure how to avoid this issue because I can't control how the value source is instantiated. 
ValueSource
has no the same APIs than the tasks where you can pass whatever you want as 
varargs
.
looks like it is not possible to access to the valueSource instance because it returns the 
Provider<String>
directly, so I will have to pass everything into it. Should be great I can somehow teach Gradle how a third party class can be serialized or something so
c
a BuildService is a singleton and could hold your jgit instance.
j
yeah, I am moving everything into it, it is being a hard refactor because I was using a sealed interface and for some reason it is not serializable with value sources and build services so I have to move everything to property of primitive types
c
that sounds possibly more complicated than it needs to be. The internals of a build service can do whatever is needed - serializable types etc not required, those are only at the edges of the Gradle API.
j
yep... anyways I hope the jgit cache implementation can be enough and I haven't to cache every git operation I am doing
looks like the new implementation is not working correctly with configuration cache and project isolation
with the new implementation now the time per project is almost zero, but it is taking 3 minutes instead of 1 (it should take less than 20 secs)
Copy code
TIME: 1
TIME: 0
TIME: 0
...
time to try the profiler I think
c
is that time in seconds? perhaps do it in ms, those ‘zero second’ ones can add up. and, yes, the profiler will help.
j
millis
Now the problem has be moved to other place
TIME: 2070 TIME: 1715 TIME: 1730 TIME: 1663 TIME: 1658 TIME: 1750 TIME: 1728 ... That is the time to configure each project, and it is taking so long due to my plugin
Copy code
BUILD SUCCESSFUL in 1m 41s
I think I know what is the problem
Previously, I haven't to call 
get()
to the build service in configuration
and I can't use it as provider, because git is not serializable so I am totally blocked if that is the problem
c
all you need is the version string - can do 
gitTagBuildService.map { it.git // … call version calculation}
to get a Provider<String> without exposing 
git
.
it gets awkward passing the Project around - suggest passing in just what is required, not the Project object.
j
I am abstracting from the plugin class to register everything there, that is the reason I pass the project, to avoid having a large class
c
passing the Project around is a code smell, should look at alternate designs.
j
That was a nice try, because with the mapping I can use the git provider inside 
providers.of
so it should be lazy, but it is still taking 
1m 33.358s
. I am going to retry with the old version of the plugin again
passing the Project around is a code smell, should look at alternate designs
I think in that register I can pass objects or providers and not the entire git dir
Old implementation:
Copy code
BUILD SUCCESSFUL in 12s
129 actionable tasks: 129 executed
maybe is the cost of building build services and value sources?
c
unlikely, though perhaps one of the serializable parameters is large/expensive.
j
maybe the list of commits
c
all that could be hidden in the BuildService w/o needing to pass through a Gradle API, as it’s internal implementation.
j
how? I am passing the list as parameter because I can't pass git instance as it is not serializable
c
on you build service expose a method “gimmeDaVersion( … )” that takes the minimum parameters necessary. Inside that method either return a cached version or do the git-fu to determine the version.
j
well technically I am calculating the version in the value source because it needs to be different in each project, and I can't pass anything to it if it is not via parameters, right?
build service is only caching the git instance across projects
c
that method can take normal Kotlin parameters. For example, pass in the project dir and do the work from there.
that could further be cached (in a map) in the build service, storing the version for a given project directory.
doing something similar to resolve installed tooling versions - pass in a version, the build service finds (or installs) the right tooling, caching the result.
j
obtains method?
or you mean creating a custom method in build service and somehow calculates it all versions, but not sure how the value source will recalculate it
I think I have already broken the config cache because something "similar to that"
c
on your build listener add a method 
getVersion( dir : File ) : String
; inside there do all the work. No need for ValueSource as the build service is the provider. invoke via `buildService.map { it.getVersion( project.projectDir ) }`to get a Provider<String>.
j
but the value source is which is recomputed every time due to its nature, if I don't use it, when everything is cached, the version will no change if, for example, git status has changed
c
perhaps that is still needed on the front-end, solely calling the build service to do the work.
j
the problem is the value source 100% but I am not sure how can I invalidate the configuration cache based on git status, I tried what Vampire suggested about to set 
.git
dir as an input in the extension, but the config cache was not invalidated even when it was changing
it is the serialization for sure
Looks like the cache of the implementation of the jgit instance cache is not working withing the value source, so I have to cache everything except inside the build service, the calculation with cached lists shouldn't be a problem
Good news are the project build correctly in terms of build speed. Bad news are configuration cache and project isolation is broken (this last one I don't understand why because I am not accessing to a different project)
Copy code
BUILD SUCCESSFUL in 9s
156 actionable tasks: 156 executed
v
afair it should tell you why in the CC HTML report
j
where is that report generated?
v
The link is printed after each run
j
Ah, because the assertion was falling, it was not being printed
well, the problem indeed it is not failing the configuration cache, the entry is not being discarded so the configuration cache "is working"
7 Views
Open in Slack
PreviousNext
Powered by