Ralph Wadner Blaise
05/27/2024, 9:10 PMCaused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.persistence.PossibleInconsistentStateException: org.apache.flink.kubernetes.shaded.io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: PUT at: https://172.1.0.1:443/api/v1/namespaces/flink-dev/configmaps/flink-v-119-cluster-config-map. Message: Operation cannot be fulfilled on configmaps "flink-v-119-cluster-config-map": the object has been modified; please apply your changes to the latest version and try again. Received status: Status(apiVersion=v1, code=409, details=StatusDetails(causes=[], group=null, kind=configmaps, name=flink-v-119-cluster-config-map, retryAfterSeconds=null, uid=null, additionalProperties={}), kind=Status, message=Operation cannot be fulfilled on configmaps "flink-v-119-cluster-config-map": the object has been modified; please apply your changes to the latest version and try again, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=Conflict, status=Failure, additionalProperties={}).
P.S My annotations for my flink deployment is as follows:
high-availability.storageDir: 'file:///flink-data/ha'
high-availability.type: kubernetes
jobmanager.memory.jvm-metaspace.size: 1024m
state.checkpoints.dir: 'file:///flink-data/checkpoints'
state.savepoints.dir: 'file:///flink-data/savepoints'
taskmanager.numberOfTaskSlots: '20'Gyula Fóra
05/28/2024, 3:19 AMRalph Wadner Blaise
05/28/2024, 3:35 AMRalph Wadner Blaise
05/28/2024, 3:45 AMapiVersion: flink.apache.org/v1beta1
kind: FlinkDeployment
metadata:
name: flink-v-119
spec:
image: flink:1.19
flinkVersion: v1_19
flinkConfiguration:
taskmanager.numberOfTaskSlots: "20"
state.savepoints.dir: file:///flink-data/savepoints
state.checkpoints.dir: file:///flink-data/checkpoints
high-availability.type: kubernetes
high-availability.storageDir: file:///flink-data/ha
jobmanager.memory.jvm-metaspace.size: 1024m
env.java.opts.jobmanager:
-Dkubernetes.max.concurrent.requests=1000
jobManager:
replicas: 2
resource:
memory: 2048m
cpu: 1
taskManager:
resource:
memory: 2048m
cpu: 1
podTemplate:
spec:
containers:
- name: flink-main-container
volumeMounts:
- name: flink-vol
mountPath: /flink-data
- name: "kv-mongocloud-xxxxx"
mountPath: "/var/secrets/kv-mongocloud-xxxxxx"
readOnly: true
- name: "kv-k8s-xxxxx"
mountPath: "/var/secrets/kv-k8s-xxxxx"
readOnly: true
- name: "kv-kafka-xxxxxxx"
mountPath: "/var/secrets/kv-kafka-xxxxxxxx"
readOnly: true
envFrom:
- configMapRef:
name: flink-configmap
- secretRef:
name: flink-mongodb-secrets
- secretRef:
name: flink-secrets
- secretRef:
name: flink-kafka-secrets
volumes:
- name: "kv-mongocloud-xxxxxxx"
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "release-name-kv-mongocloud-xxxxx"
- name: "kv-k8s-stg-001-inline"
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "release-name-kv-k8s-xxxxx"
- name: "kv-kafka-xxxxxxxxxxxxxxx"
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: "release-name-kv-kafka-xxxxxxx"
- name: flink-vol
persistentVolumeClaim:
claimName: flink-pvc-stg
serviceAccount: flinkGyula Fóra
05/28/2024, 10:35 AMRalph Wadner Blaise
05/28/2024, 2:21 PM---
# Source: flink-sql-jobs/templates/flink-session-job.yaml
apiVersion: flink.apache.org/v1beta1
kind: FlinkSessionJob
metadata:
name: job-offer-accepted-forwarder
spec:
deploymentName: flink-v-119
restartNonce: 1700
job:
jarURI: https://artifactory.xxx.com:443/artifactory/xxx.maven.ado.dev.local/sql-runner-1.5.jar
args: ["--fname=flink-lead-offer-accepted-forwarder-1.7.sql"]
parallelism: 2
upgradeMode: savepoint
state: running
allowNonRestoredState: true
---
# Source: flink-sql-jobs/templates/flink-session-job.yaml
apiVersion: flink.apache.org/v1beta1
kind: FlinkSessionJob
metadata:
name: job-knative-to-sidekick-forwarder
spec:
deploymentName: flink-v-119
restartNonce: 1700
job:
jarURI: https://artifactory.xxx.com:443/artifactory/xxx.maven.ado.dev.local/sql-runner-1.5.jar
args: ["--fname=knative-to-sidekick-forwarder-1.7.sql"]
parallelism: 1
upgradeMode: savepoint
state: running
I have nothing else deployed, just the FlinkDeployment above and the FlinkSessionJobs that I also have an example above. Is there something one with either of those yaml declarations?Ralph Wadner Blaise
05/28/2024, 9:21 PMGyula Fóra
05/29/2024, 2:58 PMRalph Wadner Blaise
05/29/2024, 3:06 PMhigh-availability.kubernetes.leader-election.lease-duration: 300s
high-availability.kubernetes.leader-election.renew-deadline: 90s
high-availability.kubernetes.leader-election.retry-period: 15s
Thanks for you help, please let me know if should bring any of these values down.
Lastly, I am now having an issue with pulling stuff from the web for example: https://artifactory.xxx.com:443/artifactory/xxx.maven.ado.dev.local/sql-runner-1.5.jar. Should that be working in HA?Ralph Wadner Blaise
05/30/2024, 6:53 PM