https://flink.apache.org/ logo
Join Slack
Powered by
:wave: Hello, team! I have `FlinkDeployments` for...
# random
h
馃憢 Hello, team! I have 
FlinkDeployments
for one process the migration using 
flink-kubernetes-operator
. This deployments was called from one job of jenkins manually, this FlinkDeployment runing during few minutes and finish, but when they finish this FlinkDeployment your status stays at Runing, this is correct? How do I make it delete automatically?
g
There is no automatic deletion of FlinkDeployments in terminal states . This is to support some use cases like backfilling etc You can check the jobstatus and write a simple logic to delete them manually
h
ok, thanks @Gyula F贸ra
Hi @Gyula F贸ra, I just updated my 
flink-kubernetes-operator v1.2
and I just realized that my pods after finishing the job, do not shut down, how can I force them to shut down when finished?
m
@Hector Malpica (Miuler) Please follow the Flink slack community rules and don't ping people directly
馃憤 1
h
I see that they updated this, I don't see a way to force the shutdown, or is there any other way? I have batch tasks that have to finish when completing the task, how would I do in these cases?
g
This config only affects the JobManager pod, do you see taskmanagers also staying around?
can you share your 
yaml
?
h
yes but I launch new jobs every night, I know, new jobmanagers would be created and they would never be closedyes, but
yes
g
if you delete the FlinkDeployment after it鈥檚 finished the jobmanager will be deleted. There is a pending ticket to improve this behaviour for 1.3.0: https://issues.apache.org/jira/browse/FLINK-29609
The current operator behaviour is more optimized for streaming jobs
馃憖 1
h
Copy code
apiVersion: <http://flink.apache.org/v1beta1|flink.apache.org/v1beta1>
kind: FlinkDeployment
metadata:
  namespace: flink-wappe-pipeline
  #name: migration-cosmosdb
  name: migrate-cosmosdb-collection1
spec:
  image: <http://xxxxxx.azurecr.io/migration-cosmosdb:latest|xxxxxx.azurecr.io/migration-cosmosdb:latest>
  imagePullPolicy: Always
  serviceAccount: flink
  flinkVersion: v1_15
  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "2"
  jobManager:
    replicas: 1
    resource:
      memory: "4096m"
      cpu: 1
  taskManager:
    resource:
      memory: "4096m"
      cpu: 1
  ingress:
    template: "<http://aks-xxxxx-ingress.eastus2.cloudapp.azure.com/{{namespace}}/{{name}}(/%7C$)(.*)|aks-xxxxx-ingress.eastus2.cloudapp.azure.com/{{namespace}}/{{name}}(/|$)(.*)>"
    className: "nginx"
    annotations:
      <http://nginx.ingress.kubernetes.io/rewrite-target|nginx.ingress.kubernetes.io/rewrite-target>: "/$2"
  podTemplate:
    spec:
      containers:
        - name: flink-main-container
  job:
    jarURI: local:///migration-cosmosdb.jar
    parallelism: 1
    upgradeMode: stateless
    args:
      - --jobName=migracert-cosmosdb-shoppingexperiencescore
      - --sourceDbUri=<mongodb://bd-xxxxx>:xxxxxx@bd-yyyy.documents.azure.com:10255/?ssl=true&retrywrites=false&replicaSet=globaldb
      - --targetDbUri=jdbc:<postgresql://psql-xxxxxx.postgres.database.azure.com:5432/DB-yyyyy?user=zzzzz&password=qqqqqqqq&sslmode=require>
      - --databaseName=bd-xxxxxxxxxx
      - --runner=FlinkRunner
      - --collections=collection1
The current operator behaviour is more optimized for streaming jobs
I'm already realizing, this is going to be a problem 馃槩
any suggestion?
g
I already wrote you my suggestions. 1. You can delete the deployments after entering a finished state 2. Wait for the fix in operator 1.3 3. Fix it yourself and help contribute the change 馃檪
馃槅 1
h
applies only for versions greater than 1.4, I think I will go down to 1.14 then
ok, Although my image is compiled with flink 
1.15
, if I put in the FlinkDeployment 
flinkVersion: v1_14
, it already works, and deletes the jobmanager. Thanks Gyula, I will be attentive to the issue FLINK-29609
Open in Slack
PreviousNext
Powered by