Hi! I am trying to start a Flink cluster on Kubern...
# troubleshooting
j
Hi! I am trying to start a Flink cluster on Kubernetes in Azure with the Kubernetes Operator and I would like to use Azure Blob storage for the save- and checkpoints. I have been trying to read the documentation but I am not sure if I am going to need an additional JAR for this or not. This page: https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/azure/ mentions that you need a JAR for Shaded Hadoop Azure Blob Storage but is that the same as the 'regular' Blob Storage? Or is the regular blob storage supported out of the box? 🀞
m
I would love to get it clarified, since https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/overview/ mentions
Azure Blob Storage is supported by flink-azure-fs-hadoop and registered under the wasb(s):// URI schemes. The implementation is based on the Hadoop Project but is self-contained with no dependency footprint.
a
TBH, if I remember well, I could not deploy properly without this plugin copied from opt/ to plugins/. But flink image already had jar file. I needed just to create own dockerfile like:
Copy code
FROM flink:1.16.2-java11

.... some mu stuff

RUN mkdir -p /opt/flink/plugins/azure-fs-hadoop
COPY ${FLINK_DIR}/opt/flink-azure-fs-hadoop-1.16.2.jar /opt/flink/plugins/azure-fs-hadoop/
πŸ™ 1
I copied in this file also my jar file, so I needed dockerfile anyway