Hi all, I'm having some problems with setting up Flink with Hive Catalog. Im facing a LinkageError while submitting my SQL job (FLink 1.16.1) That tries to write to S3. The problem Does not occur when I'm trying to write into local path. I've created the HADOOP_CLASSPATH env variable that points to hadoop 3.1.3 / This is the exception:

Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'void org.apache.hadoop.util.SemaphoredDelegatingExecutor.(com.google.common.util.concurrent.ListeningExecutorService, int, boolean)' the class loader org.apache.flink.util.ChildFirstClassLoader @2486925f of the current class, org/apache/hadoop/fs/s3a/S3AFileSystem, and the class loader 'app' for the method's defining class, org/apache/hadoop/util/SemaphoredDelegatingExecutor, have different Class objects for the type com/google/common/util/concurrent/ListeningExecutorService used in the signature (org.apache.hadoop.fs.s3a.S3AFileSystem is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @2486925f, parent loader 'app'; org.apache.hadoop.util.SemaphoredDelegatingExecutor is in unnamed module of loader 'app')
	at org.apache.hadoop.fs.s3a.S3AFileSystem.create(S3AFileSystem.java:769)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1118)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1098)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:987)
	at io.delta.storage.S3SingleDriverLogStore.write(S3SingleDriverLogStore.java:299)

Any idea what can be causing this problem? The same job is workign fine when trying to write into local path, so Im guessing that somethign with s3 plugin, some dependency issue. Other job, that does not use Hive catalog can write to S3 on the same cluster.

Which Java version are you using?

my job has

<target.java.version>11</target.java.version>

but Delta connector is 1.8

But might your local JDK be 17 by any chance?

no, no its 11

Hmmm. I've seen this error only with JDK 17 to be honest

hm, but how that ticket would be related to Flink job? via flink-sql-connector-hive-2.3.9_2.12-1.16.0.jar?

OK

hehe 🙂

You’re welcome!