Hi folks, :wave: We're using Flink Operator v1.12 ...
# troubleshooting
x
Hi folks, 👋 We're using Flink Operator v1.12 with Flink 2.0, and looking for some help understanding and debugging a mismatch between the r*econciled upgradeMode* and what we’ve configured in our FlinkDeployment spec. Any guidance on this would be greatly appreciated🙏🏻 Thanks!
Finding 1 -- From the flinkdeployment spec, we noticed that the Last Stable Spec/Last Reconciled Spec has upgradeMode being savepoint, even though job configuration indeed has upgrade mode being last-state (Results below are all returned from
k describe flinkdeployment
)
Copy code
Flink Version:                                                          v2_0
  Image:                                                                  docker-fs-identity-release.dr-uw2.adobeitc.com/identity-data-prep:140fe51148a3642b7d1525164181deb45e9f3372
  Image Pull Policy:                                                      Always
  Job:
    Args:
      identityDataPrepExecutionEnv
    Entry Class:   com.adobe.aep.fns.identity.data.prep.app.Main
    Jar URI:       local:///opt/flink/usrlib/all.jar
    Parallelism:   1
    State:         running
    Upgrade Mode:  last-state
  Job Manager:
    Replicas:  1
    Resource:
      Cpu:     2
      Memory:  8g
and
Copy code
Last Reconciled Spec:      {"spec":{"job":{"jarURI":"local:///opt/flink/usrlib/all.jar","parallelism":1,"entryClass":"com.adobe.aep.fns.identity.data.prep.app.Main","args":["identityDataPrepExecutionEnv"],"state":"suspended","savepointTriggerNonce":null,"initialSavepointPath":null,"checkpointTriggerNonce":null,"upgradeMode":"savepoint","allowNonRestoredState":null,"savepointRedeployNonce":null,"autoscalerResetNonce":null},"restartNonce":1751972266, .....}

Last Stable Spec:          {"spec":{"job":{"jarURI":"local:///opt/flink/usrlib/all.jar","parallelism":1,"entryClass":"com.adobe.aep.fns.identity.data.prep.app.Main","args":["identityDataPrepExecutionEnv"],"state":"running","savepointTriggerNonce":null,"initialSavepointPath":null,"checkpointTriggerNonce":null,"upgradeMode":"savepoint","allowNonRestoredState":null,"savepointRedeployNonce":null,"autoscalerResetNonce":null},"restartNonce":1751969519, ....}
Finding 2 -- Savepoint/checkpoint timestamp/history are also missing
Copy code
Job Status:
    Checkpoint Info:
      Last Periodic Checkpoint Timestamp:  0
    Job Id:                                42ab796df84427a35779ab94184d2430
    Job Name:                              identity-data-prep
    Savepoint Info:
      Last Periodic Savepoint Timestamp:  0
      Savepoint History:
    Start Time:              1751969599559
    State:                   CANCELED
    Update Time:             1751972272559
    Upgrade Savepoint Path:  wasbs://streamingestflink@stageva7identity.blob.core.windows.net/_realtime_flink/dcs/42ab796df84427a35779ab94184d2430/chk-12
P.S. We did notice that the Flink Operator 1.12 release notes mention a fix for a broken last-state upgrade mode — tracked in FLINK-37571. We've already upgraded the operator, but are wondering if the strange behavior we observed (e.g., fallback to savepoint or stateless mode) was related to this bug.