Thanks for comprehensive answer!
Yes, I'm referring to the build script classpath.
Adding or removing new entry is the only problem I have with custom solution I've came up with.
All build cache is lost on every change in buildSrc, because build script classpath was changed. Essentially causing a clean build.
I had a hope, that Version Catalog somehow deals with this problem.
P.S. By "build cache" I mean not Gradle Build Cache, maybe it's called project cache (/build).