Mali
03/08/2023, 3:12 PMapiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkDeployment
metadata:
name: test
namespace: flink
spec:
ingress:
template: "{{name}}.<http://test.com|test.com>"
className: "nginx"
annotations: {}
image: <private pyflink repo>
imagePullPolicy: "Always"
flinkVersion: v1_16
flinkConfiguration:
taskmanager.numberOfTaskSlots: "1"
serviceAccount: flink
jobManager:
podTemplate:
apiVersion: v1
kind: Pod
metadata:
name: pod-template
spec:
serviceAccount: flink
containers:
# Do not change the main container name
- name: flink-main-container
envFrom:
- secretRef:
name: flink-secret
resource:
memory: "2048m"
cpu: 1
taskManager:
resource:
memory: "2048m"
cpu: 1
replicas: 1
---
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkSessionJob
metadata:
name: rabbitmq-to-kafka
namespace: flink
spec:
deploymentName: test
job:
jarURI: local:///opt/flink/lib/flink-sql-connector-rabbitmq-1.16.0.jar;local:///opt/flink/lib/flink-sql-connector-kafka-1.16.0.jar
entryClass: "org.apache.flink.client.python.PythonDriver"
args: ["-pyclientexec", "/usr/local/bin/python3", "-py", "/opt/flink/usrlib/rabbitmq_to_kafka.py"]
parallelism: 4
upgradeMode: stateless
---
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkSessionJob
metadata:
name: kafka-to-s3-raw
namespace: flink
spec:
deploymentName: test
job:
jarURI: local:///opt/flink/lib/flink-sql-connector-rabbitmq-1.16.0.jar;local:///opt/flink/lib/flink-sql-connector-kafka-1.16.0.jar
entryClass: "org.apache.flink.client.python.PythonDriver"
args: ["-pyclientexec", "/usr/local/bin/python3", "-py", "/opt/flink/usrlib/kafka_to_s3_raw.py"]
parallelism: 4
upgradeMode: stateless
I am getting ” resource id: ResourceID{name=‘kafka-to-s3-raw’, namespace=‘flink’}, version: 6850788} failed.
org.apache.flink.kubernetes.operator.exception.ReconciliationException: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme ‘local’. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded. For a full list of supported file systems, please see https://nightlies.apache.org/flink/flink-docs-stable/ops/filesystems/.”
It is working well when i put jobURI inside of deployment. I tried different alternatives but didn’t figure it out. Can you help me about this error ?
Many thanks.kingsathurthi
03/09/2023, 4:18 AMkingsathurthi
03/09/2023, 4:19 AMapiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkSessionJob
metadata:
name: basic-session-job-only-example
spec:
deploymentName: basic-session-deployment-only-example
job:
jarURI: <https://repo1.maven.org/maven2/org/apache/flink/flink-examples-streaming_2.12/1.15.3/flink-examples-streaming_2.12-1.15.3-TopSpeedWindowing.jar>
parallelism: 4
upgradeMode: statelessMali
03/09/2023, 5:53 AMMali
03/09/2023, 5:55 AMkingsathurthi
03/09/2023, 6:14 AM---
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkSessionJob
metadata:
name: rabbitmq-to-kafka
namespace: flink
spec:
deploymentName: test
job:
jarURI: <https://registry.com/job_1.jar>
entryClass: "org.apache.flink.client.python.PythonDriver"
args: ["-pyclientexec", "/usr/local/bin/python3", "-py", "/opt/flink/usrlib/rabbitmq_to_kafka.py"]
parallelism: 4
upgradeMode: stateless
---
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkSessionJob
metadata:
name: kafka-to-s3-raw
namespace: flink
spec:
deploymentName: test
job:
jarURI: <https://registry.com/job_2.jar>
entryClass: "org.apache.flink.client.python.PythonDriver"
args: ["-pyclientexec", "/usr/local/bin/python3", "-py", "/opt/flink/usrlib/kafka_to_s3_raw.py"]
parallelism: 4
upgradeMode: statelessMali
03/09/2023, 6:15 AM