Hey everyon I attach the java errors I get from trying to ru Apache Flink #troubleshooting

Hey everyon, I attach the java errors I get from t...

Eugenio Gastelum

07/10/2023, 8:05 PM

Hey everyon, I attach the java errors I get from trying to run the python UDF file in case it helps. I've attached it as a comment on this thread

Eugenio Gastelum

07/10/2023, 8:06 PM

Here they are

errors.txt

Hong Teoh

07/12/2023, 1:48 PM

Heya - which Flink version are you using? This issue was seen previously here and should be fixed here https://issues.apache.org/jira/browse/FLINK-29324

Eugenio Gastelum

07/12/2023, 3:27 PM

@Hong Teoh thanks! I have tried it on flink

1.15.2

and then I tried

1.17.1

Eugenio Gastelum

07/12/2023, 3:27 PM

Well, those are my pyflink library versions

Hong Teoh

07/12/2023, 3:28 PM

ok cool, but which SQL jar are you loading for the Kinesis connector?

Eugenio Gastelum

07/12/2023, 3:29 PM

Oh okay, this one

flink-sql-connector-kinesis-1.15.2.jar

Hong Teoh

07/12/2023, 3:29 PM

kk, you will need to use the 1.15.3 or 1.15.4 one!

Eugenio Gastelum

07/12/2023, 3:29 PM

Ok ok thanks! I'll try it

Eugenio Gastelum

07/12/2023, 4:02 PM

I've just tried with

1.15.4

and

1.16.1

but I got the same error

Eugenio Gastelum

07/12/2023, 4:03 PM

Those are the

flink-sql-connector-kinesis-1.15.4.jar

and

flink-sql-connector-kinesis-1.16.1.jar

Hong Teoh

07/12/2023, 4:34 PM

hmm thats weird

Hong Teoh

07/12/2023, 4:34 PM

Whats the full stack trace of the error?

Eugenio Gastelum

07/12/2023, 4:35 PM

It's this

error_flink.txt

Eugenio Gastelum

07/12/2023, 4:37 PM

I've also simplified the flow, the UDF just returns the input now

flinkAnalyzer_udf.py

Eugenio Gastelum

07/12/2023, 4:38 PM

And the line in that python where it fails is the one where it leaves the job running, the

result_table.wait()

line

Eugenio Gastelum

07/12/2023, 4:39 PM

That specific stack trace came from using the python library

apache-flink 1.17.1

and the sql connect jar

flink-sql-connector-kinesis-1.16.1.jar

Hong Teoh

07/12/2023, 4:46 PM

Hmm, this seems like a different error TBH… The previous stack trace has

Copy code

Suppressed: java.lang.NullPointerException
                at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.close(FlinkKinesisConsumer.java:421)
                at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41)
                at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.close(AbstractUdfStreamOperator.java:115)
                at org.apache.flink.streaming.api.operators.StreamSource.close(StreamSource.java:124)
                at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:163)
                at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.closeAllOperators(RegularOperatorChain.java:125)
                at org.apache.flink.streaming.runtime.tasks.StreamTask.closeAllOperators(StreamTask.java:1043)
                at org.apache.flink.util.IOUtils.closeAll(IOUtils.java:255)
                at org.apache.flink.core.fs.AutoCloseableRegistry.doClose(AutoCloseableRegistry.java:72)
                at org.apache.flink.util.AbstractAutoCloseableRegistry.close(AbstractAutoCloseableRegistry.java:127)
                at org.apache.flink.streaming.runtime.tasks.StreamTask.cleanUp(StreamTask.java:951)
                at org.apache.flink.runtime.taskmanager.Task.lambda$restoreAndInvoke$0(Task.java:934)
                at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952)
                at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:934)
                ... 3 more

Open in Slack

Previous Next