This message was deleted.

The message means that some annotation processor is generating a resource 'META-INF/BenchmarkList in CLASS_OUTPUT', but it is not an incremental annotation processor. Annotation processor has to follow some rules to be incremental with Gradle, see: https://docs.gradle.org/current/userguide/java_plugin.html#sec:incremental_annotation_processing

There seems to be only a single testAnnotationProcessor in the build.gradle file:

testImplementation group: 'org.openjdk.jmh', name: 'jmh-core', version: '1.35'
    testAnnotationProcessor group: 'org.openjdk.jmh', name: 'jmh-generator-annprocess', version: '1.35'

Could this be causing problems somehow?

It looks like it, but I don't know how

jmh-generator-annprocess

works internally. I see that if you use jmh-plugin then incremental compilation works. Maybe that is a good option for you. Note that a plugin configures a new sourcset (folder) for you called jmh

Thanks alot, I'll take a look!

Fwiw, looks like jmh-generator-annprocess should be declared aggregating, not isolating: https://github.com/openjdk/jmh/blob/master/jmh-core/src/main/java/org/openjdk/jmh/generators/core/BenchmarkGenerator.java#L188