Hi Team, I am using the kinesis connector with Fli...
# troubleshooting
k
Hi Team, I am using the kinesis connector with Flink and getting the following error:
Copy code
t=37719).
java.lang.NoClassDefFoundError: Could not initialize class org.apache.flink.kinesis.shaded.com.amazonaws.auth.internal.AWS4SignerUtils
	at org.apache.flink.kinesis.shaded.com.amazonaws.auth.internal.AWS4SignerRequestParams.<init>(AWS4SignerRequestParams.java:112) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.auth.AWS4Signer.sign(AWS4Signer.java:233) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1320) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:3169) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:3136) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:3125) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.executeDescribeStream(AmazonKinesisClient.java:942) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.AmazonKinesisClient.describeStream(AmazonKinesisClient.java:911) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.describeStream(KinesisProxy.java:645) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.lookupStreamArn(KinesisProxy.java:502) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardsOfStream(KinesisProxy.java:472) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.proxy.KinesisProxy.getShardList(KinesisProxy.java:345) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.internals.KinesisDataFetcher.discoverNewShardsToSubscribe(KinesisDataFetcher.java:926) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.connectors.kinesis.FlinkKinesisConsumer.run(FlinkKinesisConsumer.java:324) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:181) ~[flink-dist-1.20.0-amzn-2.jar:1.20.0-amzn-2]
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78) ~[flink-dist-1.20.0-amzn-2.jar:1.20.0-amzn-2]
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:340) ~[flink-dist-1.20.0-amzn-2.jar:1.20.0-amzn-2]
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.NoClassDefFoundError: org/joda/time/format/DateTimeFormat [in thread "Legacy Source Thread - Source: Custom Source -> Map (1/1)#1"]
	at org.apache.flink.kinesis.shaded.com.amazonaws.auth.internal.AWS4SignerUtils.<clinit>(AWS4SignerUtils.java:27) ~[flink-connector-kinesis-5.0.0-1.20.jar:5.0.0-1.20]
	... 25 more
The error is caused due to
Exception java.lang.NoClassDefFoundError: org/joda/time/format/DateTimeFormat
. Is there a reason for the joda time dependency to not be present in flink-connector-kinesis-5.0.0-1.20.jar ? Why wasn’t it shaded along with the other dependencies.
a
I would try the uber jar dependency first
flink-sql-connector-kinesis-5.0.0-1.20.jar
That should bundle all dependencies
k
oh ok makes sense. I will try that.