hey guys, I have a question about plugins mechanism ,to be specific , hadoop-s3 plugin

1.  My APP is runing on kubernetes managed by flink kubernetes operator, I use ENV ENABLE_BUILT_IN_PLUGINS to link hadoop-s3 plugin to where it supposed to be, and I exec into the pod ,confirmed iflink-s3-fs-hadoop-1.15.2.jar is in plugins/hadoop-s3 folder.

2.  My APP will read proto records from kafka, so I wrote a deserializer which will call hadoop releated class funciton ,but not directly ,in a very deep chain.

3.  However , it report NoClassFoundError org/apache/hadoop/conf/Configuration ,and the error trace back to deserializer  

4.  Then I append hadoop dependency into my application pom ,and package into a fat jar, guess what, it worked

So can somebody please explain this to me ? I read a lot of times of Flink DOC about plugins and classloader ,still no clue

The

hadoop-s3-plugin

is stricly for connecting to S3; it doesn't provide you with Hadoop on your classpath. See https://nightlies.apache.org/flink/flink-docs-master/docs/dev/configuration/advanced/#hadoop-dependencies on how to use Flink with Hadoop

Thanks, Visser!! That explains a lot! I got another question. According to Flink doc.

there is a hierarchy of typically two ClassLoaders: (1) Java's _application classloader_,  (2) the dynamic _plugin/user code classloader_.The dynamic ClassLoader has the application classloader as its parent.And plugin classloader is dedicated per plugin.

By default, Flink inverts classloading order, child first then parent. But there are exceptions, org.apache.hadoop is one of them. So for example, when I sink files into S3 which will use classes in org.apache.hadoop. In order to do that , I have to copy hadoop-s3 jar into plugins/hadoop-s3 package . Right now ,there are only one hadoop jar in my environment. So when the app starts, The parent classloader will first look into classpath , and find no hadoop jar there. Then it will call its child classloader. The question is which class loader will it call, the user code dynamic classloader or the hadoop-s3 plugin dynamic classloader? sure its the latter one, but how does he konw? what if I also package hadoop jar into my app jar as a fat jar. what will happen then?

Plugins have their own dedicated classloader

Thanks Visser, I know plugins have their own dedicated classloader. But my question is about classloader loading order. As for classes in org.apache.hadoop , they are always loaded by parent classloader first, if there are no such classes in parent classloader classpath, it will go to child classloader. But there are multiple child loaders, user code classloader, many plugins classloader, which one will it use and how?