Hi all. In Hibernate ORM we're having trouble with...
# community-support
c
Hi all. In Hibernate ORM we're having trouble with compile times, because the incremental compilation is too coarse grained. It looks like this was reported in the past, but unfortunately, there is no link to a GitHub issue where I could track the progress for that. I tried to search for some keywords but couldn't find anything yet. Does anyone of you have an idea if this problem is tracked somewhere?
I debugged the SelectiveCompiler in Gradle a bit to understand how this works, and it seems to me, that this coarse grained-ness is sort of by design. The use of ClassSetAnalysis, which is used to determine dependent classes, assumes "the worst case" i.e. that a file changed fundamentally. I was wondering if the dependent classes calculation could happen after compilation of the changed source files. That way, a the dependent classes calculation could take into account if the changes were actually "visible" to dependent classes
I could imagine writing a diff utility for two class files that computes lists of changes that affect • subclasses • usages within the same package • usages outside of the package but within same module • usages outside of the package and outside of the module based on that information, the amount of dependent classes could be reduced significantly in our case and probably also for many other users out there. compilation for dependent classes would then be triggered separately. Not sure if you'd want to do separate compile transactions for the various passes or do all of that within one compile transaction with multiple compiler executions
a
I was wondering if the dependent classes calculation could happen after compilation of the changed source files.
We calculate which classes we need to recompile after the file is changed. The problem is, that we don't analyse what in source file changed: did ABI change (e.g. method signature) or just implementation (e.g. method body)
v
It looks like this was reported in the past, but unfortunately, there is no link to a GitHub issue where I could track the progress for that.
It should be fixed in 6.0 actually
Found the related issue at https://github.com/gradle/gradle/issues/7401 and the PR that implemented it at github.com/gradle/gradle/pull/10419
c
The problem is, that we don't analyse what in source file changed: did ABI change (e.g. method signature) or just implementation (e.g. method body)
Yeah, that's exactly the problem we're running into
we often change stuff in an ABI compatible manner, but then have to suffer excessive compilation times
šŸ˜ž 1
a
Note that this is a problem only inside the same source set. We can detect such changes between projects though. So if you have a lot of classes in one project (or source set) then current solution is to split it in smaller projects to get better performance.
c
Yeah, I understand that, but unfortunately, that's not possible
a
Yeah, it's easier said than done...
c
so do you have a GitHub issue for this already?
a
Hmmm, it should be somewhere
v
c
Yeah, that looks like what I need šŸ™‚
thanks
assuming I wanted to work on this, what would be the process?
ā¤ļø 1
I guess we'd first have to discuss a design before I do anything and present you a PR
a
Yes, it would be good to discuss it first. This is very complex part of the code, so it means also the team needs to spend some time reviewing changes too. Which means it might happen they don't have time to work on it due to other priorities
c
I understand that. As an OSS developer, I am facing the same challenges in the Hibernate ORM community. Unfortunately, this issue is really destroying our productivity, so fixing it has a high priority for my team. I'll start discussing ideas on the GitHub issue then, thanks
šŸ‘ 1
s
How big is the single source set? Do the changes propagate in a way that we end up just recompiling almost everything? If you disabled incremental compilation, is it faster in the worst cases?
v
In the linked forum post it was 12K files of 15K files recompiled instead of one file recompiled as just a comment was added, but that is probably a really extreme example šŸ˜„
And that was before the improvements in 6.0
s
sure, but I was wondering about Hibernate's specific example in case there was a short term fix to alleviate pain