We are having a wierd issue with apache flink oper...
# troubleshootingh
Hussain Abbas
10/17/2023, 3:50 PMWe are having a wierd issue with apache flink operator, when ever we upgrade our flink jobs, operator would fail few jobs intermittently complaining that it could not find HA metada and manual restore is required. This is happening with almost every upgrade, we cannot see any reasonable error, except that its complaining that job cannot be stopped after taking savepoint. Any idea? Its has becoming an issue in production.
Hussain Abbas
10/17/2023, 4:50 PMOne more thing, as soon as we upgrade i see operator POD being restarted.
g
Gyula Fóra
10/17/2023, 7:56 PMWhich operator version are you using? Could it be related to https://issues.apache.org/jira/browse/FLINK-33011 ? You could pick up the fixed snapshot image or build your operator with the fix included
Gyula Fóra
10/17/2023, 7:57 PMAlternatively switching to last-state upgrade mode may help
Gyula Fóra
10/17/2023, 7:57 PMIf not please open a jira ticket with the operator logs and all the details :)
h
Hussain Abbas
10/17/2023, 7:58 PMI am using the lastest version 1.6.0, what we observed it happens with savepoint upgrade
Hussain Abbas
10/18/2023, 9:53 AMOne stupid question, is this issue only with 1.6.0 or all other previous versions?
g
Gyula Fóra
10/18/2023, 9:53 AMnot completely sure, probably due to a lib update in 1.6.0, but the patch linked in the jira is good
Gyula Fóra
10/18/2023, 9:53 AMit’s better to upgrade to 1.6.0 + the fixes then older version if you ask me
h
Hussain Abbas
10/18/2023, 11:29 AMOkay thanks for your info
2 Views