Is there a way to have sourceSets on a plugin without registering java plugin? Although I must also ensure that if the consumer applies java plugin later the build would not break.

Depending on what exactly you want you might get away with applying the

JvmEcosystemPlugin

. This gets you a

SourceSetContainer

and the variant derivation strategy and attribute schema of a JVM environment.