Is there a way to detect from a plugin when a project is being configured from the tooling API? More specifically, a way to detect when being called from Buildship would be even better.

It is most probably a bad idea to do whatever you intend to do. :-D For IntelliJ sync there is a system property you could check,

idea.sync.active

iirc. Maybe Buildship does something similar? 🤷‍♂️