Hi, I'm having a bit of an issue getting the Kafka...
# troubleshooting
t
Hi, I'm having a bit of an issue getting the Kafka sink working. I have the following dependencies: • flink-streaming-java: 2.0.0 • flink-clients: 2.0.0 • flink-connector-kafka: 4.0.0-2.0 and the following code in my
main
Copy code
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

KafkaSourceString source = KafkaSource.Stringbuilder()
        .setBootstrapServers("kafka.kafka.svc.cluster.local:9092")
        .setProperty("sasl.mechanism", "SCRAM-SHA-256")
        .setProperty("security.protocol", "SASL_PLAINTEXT")
        .setProperty("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";")
        .setTopics("input-topic")
        .setGroupId("my-group")
        .setStartingOffsets(OffsetsInitializer.earliest())
        .setValueOnlyDeserializer(new SimpleStringSchema())
        .build();

KafkaSinkString sink = KafkaSink.Stringbuilder()
        .setBootstrapServers("kafka.kafka.svc.cluster.local:9092")
        .setProperty("sasl.mechanism", "SCRAM-SHA-256")
        .setProperty("security.protocol", "SASL_PLAINTEXT")
        .setProperty("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"user\" password=\"pass\";")
        .setRecordSerializer(KafkaRecordSerializationSchema.builder()
                .setTopic("output-topic")
                .setValueSerializationSchema(new SimpleStringSchema())
                .build()
        )
        .setDeliveryGuarantee(DeliveryGuarantee.NONE)
        .build();

DataStreamString stream = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source");
stream.map(String::toUpperCase).sinkTo(sink);

env.execute("Uppercase Kafka Stream");
It compiles fine, but when I submit it to a 2.0.0 cluster running in Minikube, I get the following error:
Copy code
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Failed to execute job 'Uppercase Kafka Stream'.
Caused by: org.apache.flink.util.FlinkException: Failed to execute job 'Uppercase Kafka Stream'.
Caused by: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit ExecutionPlan.
Caused by: org.apache.flink.runtime.rest.util.RestClientException: [Internal server error., <Exception on server side: org.apache.flink.runtime.client.JobSubmissionException: Failed to submit job.
Caused by: java.lang.RuntimeException: java.io.InvalidObjectException: ReflectiveOperationException during deserialization
Caused by: java.lang.reflect.InvocationTargetException
Caused by: java.lang.IllegalArgumentException: Invalid lambda deserialization
This only happens when I add the
sinkTo
, if I replace it with a
print
then it works. Can anyone help me determine the cause? Thanks.
This seems to be related to the specific cluster. When I start a local flink server connecting to a kafka with no auth running it docker, it seems to be fine. However, when I run the cluster in Kubernetes (using the Bitnami Helm chart) I get this problem - it would still be great to get some insight on how I might debug it.
a
The
ReflectiveOperationException
makes me think it's related to JDK modularization and JVM Reflection configuration. Have a look at how the default Java Opts are configured here: https://github.com/apache/flink/blob/4c77daaf3554446e67f6b75ac18d54da84208b9a/flink-dist/src/main/resources/config.yaml#L24. Perhaps the Bitnami Helm chart doesn't include these Java Opts?
I our setup Flink logs the JVM settings during start-up. It might be a good starting point to look at those. Our's usually looks like this:
Copy code
2025-05-22 08:04:43,847 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []  JVM Options:
2025-05-22 08:04:43,848 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     -Xmx1530082096
2025-05-22 08:04:43,849 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     -Xms1530082096
2025-05-22 08:04:43,850 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     -XX:MaxMetaspaceSize=268435456
2025-05-22 08:04:43,850 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     -XX:+IgnoreUnrecognizedVMOptions
2025-05-22 08:04:43,851 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=java.base/sun.net.util=ALL-UNNAMED
2025-05-22 08:04:43,852 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=java.rmi/sun.rmi.registry=ALL-UNNAMED
2025-05-22 08:04:43,853 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
2025-05-22 08:04:43,854 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
2025-05-22 08:04:43,854 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
2025-05-22 08:04:43,855 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
2025-05-22 08:04:43,856 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
2025-05-22 08:04:43,857 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED
2025-05-22 08:04:43,857 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.lang=ALL-UNNAMED
2025-05-22 08:04:43,869 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.net=ALL-UNNAMED
2025-05-22 08:04:43,893 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.io=ALL-UNNAMED
2025-05-22 08:04:43,894 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.nio=ALL-UNNAMED
2025-05-22 08:04:43,894 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/sun.nio.ch=ALL-UNNAMED
2025-05-22 08:04:43,895 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.lang.reflect=ALL-UNNAMED
2025-05-22 08:04:43,896 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.text=ALL-UNNAMED
2025-05-22 08:04:43,896 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.time=ALL-UNNAMED
2025-05-22 08:04:43,897 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.util=ALL-UNNAMED
2025-05-22 08:04:43,898 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
2025-05-22 08:04:43,909 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
2025-05-22 08:04:43,910 INFO  org.apache.flink.runtime.entrypoint.ClusterEntrypoint        []     --add-opens=java.base/java.util.concurrent.locks=ALL-UNNAMED
t
Thanks @Andreas Bube, I’ll review the setup with the Bitnami chart