is there a way to get the Provider/ObjectFactory objects without calling them on project?

Wrong channel. But here you go https://docs.gradle.org/current/userguide/service_injection.html

oops, sorry, thanks