Hi All I am migrating to flink 1 19 0 from flink 1 17 and I Apache Flink #troubleshooting

Hi All, I am migrating to flink 1.19.0 from flink...

Vipul Jindal

08/08/2024, 10:03 AM

Hi All, I am migrating to flink 1.19.0 from flink 1.17 and I am getting this error

Copy code

Enabling required built-in plugins
Linking flink-s3-fs-presto-1.19.0.jar to plugin directory
Plugin flink-s3-fs-presto-1.19.0.jar does not exist. Exiting.

In the FlinkDeployment I have passed this env variable

Copy code

env:
              - name: ENABLE_BUILT_IN_PLUGINS
                value: flink-s3-fs-presto-1.19.0.jar

This is the base image I am using

Copy code

<http://docker.artifactory.ea.com/flink:1.19.0-java11@sha256:61355051b493c0b2784ae5755608798e7c54d9db0f8467e9e3f183e3a48e617a|docker.artifactory.ea.com/flink:1.19.0-java11@sha256:61355051b493c0b2784ae5755608798e7c54d9db0f8467e9e3f183e3a48e617a>

I verified that the flink-s3-fs-presto-1.19.0.jar was there in the plugins dir but now the container is just exiting. Kindly suggest.

D. Draco O'Brien

08/08/2024, 11:24 AM

Enable debugging or trace to get more information about the error.

D. Draco O'Brien

08/08/2024, 11:25 AM

Check the plugin JAR is located exactly where Flink expects it to be. The plugin directory path might have changed between Flink versions. For Flink 1.19, the default plugin directory is usually under

/opt/flink/plugins

, but this depends on how your Docker image is structured.

Vipul Jindal

08/08/2024, 11:26 AM

The container is exiting right after deploying it.

D. Draco O'Brien

08/08/2024, 11:26 AM

Then check permissions …

D. Draco O'Brien

08/08/2024, 11:27 AM

Check if there are any permission issues preventing Flink from accessing the plugin. The user running the Flink process inside the container needs to have read permissions for the plugin directory and its contents.

D. Draco O'Brien

08/08/2024, 11:28 AM

The ENABLE_BUILT_IN_PLUGINS environment variable should ideally contain the plugin names without the version or the .jar extension, unless explicitly required otherwise by Flink 1.19 documentation or known changes. Try setting as follows:

Copy code

env:
  - name: ENABLE_BUILT_IN_PLUGINS
    value: flink-s3-fs-presto

Verify this against the official Flink 1.19 documentation or release notes for the correct format.

D. Draco O'Brien

08/08/2024, 11:29 AM

And check flink-conf.yaml for any needed plugin settings per Flink 1.19

Vipul Jindal

08/08/2024, 11:32 AM

The document says to set the env variable value with the version and .jar extension https://nightlies.apache.org/flink/flink-docs-release-1.19/docs/deployment/resource-providers/standalone/docker/#using-filesystem-plugins

Vipul Jindal

08/08/2024, 4:59 PM

Same logs are getting printed even after changing the log levels to either TRACE or DEBUG.

Maxime Menard

11/05/2024, 8:35 PM

@Vipul Jindal I'm getting same issue with flink 1.20.0, did you have to create your own docker image to include the plugin ?

Open in Slack

Previous Next