There are definitely memory leaks somewhere. I experience this all the time with Gradle (since v7) + AGP + Kotlin, and each of those have their own issues.
I've been filing a lot of memory leak bugs, and while some have been fixed, many still haven't been, and new ones keep popping up.
In the interim I just kill the daemon every few builds.