Raman Liubimau
03/04/2025, 8:20 PMhadoop-common:3.4.1, and that was sufficient. But for this pipeline, it also requires hadoop-mapreduce-core. And that's where my troubles started.
As soon as I add the MapReduce core dependency, I get a Java linkage error:
Caused by: java.lang.LinkageError: loader constraint violation: when resolving method 'org.apache.hadoop.conf.Configuration org.apache.flink.runtime.util.HadoopUtils.getHadoopConfiguration(org.apache.flink.configuration.Configuration)' the class loader org.apache.flink.util.ChildFirstClassLoader @162afc80 of the current class, org/apache/flink/formats/parquet/avro/AvroParquetRecordFormat, and the class loader 'app' for the method's defining class, org/apache/flink/runtime/util/HadoopUtils, have different Class objects for the type org/apache/hadoop/conf/Configuration used in the signature (org.apache.flink.formats.parquet.avro.AvroParquetRecordFormat is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @162afc80, parent loader 'app'; org.apache.flink.runtime.util.HadoopUtils is in unnamed module of loader 'app')
I've tried to play around with different versions of Hadoop dependencies and tried to relocate org.apache.hadoop but without success. Finally I ran out of ideas what to try and how to fix this. I'd much appreciate if someone gives me direction.rmoff
03/05/2025, 9:17 AMRaman Liubimau
03/05/2025, 11:36 AMrmoff
03/05/2025, 11:41 AMRaman Liubimau
03/05/2025, 11:49 AMRaman Liubimau
03/05/2025, 11:51 AM