https://flink.apache.org/ logo
Join Slack
Powered by
Hei flink community, We are facing an issue with ...
# troubleshooting
o
Hei flink community, We are facing an issue with flink 1.15, 1.16 or 1.16.2 (I tried these 3 versions with same results, maybe it is more general) I am trying to test table API in local and for that I have the following dependencies in my job. See the list of dependencies at the bottom of this post. When running the tests in local I get the following error: org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath Caused by: org.apache.flink.table.api.ValidationException: Could not find any factories that implement 'org.apache.flink.table.delegation.ExecutorFactory' in the classpath. But the planner module should be there as you can see in the following dependencies. I have also checked that is there by looking at the runtime classpath for tests. Any help would be greatly appreciated. Thanks!
Copy code
implementation "org.apache.flink:flink-table-api-java:${flinkVersion}"
implementation "org.apache.flink:flink-table-runtime:${flinkVersion}"
implementation "org.apache.flink:flink-table-planner-loader:${flinkVersion}"

implementation "org.apache.flink:flink-table-api-java-bridge:${flinkVersion}"
implementation "org.apache.flink:flink-statebackend-rocksdb:${flinkVersion}"
// Add test dependencies here.
testCompileOnly "org.apache.flink:flink-streaming-java:${flinkVersion}:tests"
testImplementation "org.apache.flink:flink-test-utils:${flinkVersion}"
testImplementation "org.apache.flink:flink-test-utils-junit:${flinkVersion}"
testImplementation "org.apache.flink:flink-table-test-utils:${flinkVersion}"
testImplementation "org.apache.flink:flink-table-test-utils:${flinkVersion}"
testImplementation "org.apache.flink:flink-table-planner-loader:${flinkVersion}"
The weirdest thing is that sometimes, very seldomly works, so I guess this may be related to some classloader race condition? Has anybody experienced something similar?
m
Is there any reference to a Scala version?
o
no
we use java
kotlin as a matter of fact but not scala
we found the root cause. it turns out that we use the minicluster to instantiate flink and make our tests faster like this:
Copy code
class FlinkClusterExtension : BeforeAllCallback, AfterAllCallback {

    private val PARALLELISM = 1
    private lateinit var flinkCluster: MiniClusterWithClientResource

    override fun beforeAll(context: ExtensionContext) {
        flinkCluster =
            MiniClusterWithClientResource(
                MiniClusterResourceConfiguration.Builder()
                    .setNumberSlotsPerTaskManager(PARALLELISM)
                    .setNumberTaskManagers(1)
                    .build(),
            )

        flinkCluster.before()
    }

    override fun afterAll(context: ExtensionContext) {
        flinkCluster.after()
    }
}
if we use this annotation @ExtendWith(FlinkClusterExtension::class)
then we get classpath problems
m
I'm not familiar with Gradle or anything. But the number of dependencies that are listed there for Table API support are quite a lot...
o
if we remove it and let the StreamExecutionEnvironment.getExecutionEnvironment() get its own execution environment
no, has nothing to do with gradle I believe
it has to do with the minicluster and how it is instantiated in either way
m
No I mean that I don't immediately recognize what implementation/testimplementation means πŸ™‚
o
ah gotcha πŸ™‚
but does it sound familiar? problems of classpath with the minicluster starting up?
m
But for Table API stuff, I believe the only things that is normally included is 
flink-streaming-java
, 
flink-clients
, 
flink-table-api-java-bridge
, 
flink-table-planner-loader
and 
flink-table-runtime
which are all set to provided
And we use 
flink-test-utils
and 
flink-runtime-web
for test scopes (the latter only to make the Web UI available)
o
I ll have a deep look, thanks
2 Views
Open in Slack
PreviousNext
Powered by