Hi, I have a question regarding the Flink Kubernet...
# troubleshooting
m
Hi, I have a question regarding the Flink Kubernetes Operator. It is published on https://operatorhub.io/operator/flink-kubernetes-operator, but the latest version available is 1.6.0, which is two years old. Is there a chance to update the operator to a newer version?
g
Unfortunately the operatorhub releases weren鈥檛 maintained properly over the last year. Maybe @david radley or @James Busche could help with this.
d
@Thomas Cooper you expressed an interest to do the Red Hat version and maybe operator hub. In the past they were done together. Is this something you are likely to take on?
t
Hey, yes we (Red Hat) can take a look at this.
d
Yes - sorry I have not been keeping this up to date. Thanks @Thomas Cooper . @Tony Garrard you had some ideas as to how the current process could be simplified. Are you in a position to share.
t
@david radley I don't have access to the IBM github yet so I might need you to dm me the readme (if it differs from what it in the upstream)
d
sure
t
@Thomas Cooper dm me when you have access to IBM github
d
Thanks @Tony Garrard I am checking in what way we can share this content with Tom.
I am thinking of publishing this to the flink wiki - as that is a more appropriate place for this information to live. Or is there a better place - maybe the flink operator git repo? WDYT? I am checking with the original author to understand everything I need to bring together. @Tony Garrard @Thomas Cooper @Gyula F贸ra
t
So I have been looking at the upstream olm set up, is the IBM repo very different to what it in: https://github.com/apache/flink-kubernetes-operator/tree/main/tools/olm
t
significantly different
t
Right, is most of that IBM specific or just an update to the build/release process?
t
tbh a bit of both, especially after we gained a better understanding of building java operators
t
Yeah the upstream process could do with updating and streamlining
Can the IBM only elements be extracted easily? If not I can just keep hacking on the upstream
t
I think they could, let me see what I can do
馃檹 1
t
Would it make sense to move this process out of docker and make a GH Action workflow for the olm release? Might still need to be a manually triggered flow but might make it easier to manage?
@Gyula F贸ra Would this need to be part of the apache release process that creates the helm charts that get pushed to the apache downloads site. e.g. https://downloads.apache.org/flink/flink-kubernetes-operator-1.10.0/
The images are pushed on every release branch/tag event (from what I can see), but the OLM bundle should be once for a given release.
d
and the operatorhub
馃憤 1
j
Hi team, sorry I missed the conversation yesterday - catching up.
Interesting! I haven't been following Flink for awhile - congrats on all the releases since 1.6.0. The wiki where I kept the instructions for building the OLM bundles for both the 1.6.0 Red Hat and Operatorhub are on the IBM enterprise github, but I could move them over to the https://github.com/apache/flink-kubernetes-operator site if wiki is enabled there, or create readme/md files if needed. I don't know if the steps still work/are valid - should I see if I can create a release-1.10.0 and see if it works?
t
It might be worth it as it might have broken. We moved to a file based bundle which means the previous based catalog does not need to be present. So instead of using olm to build the bundle we use
operator-sdk generate bundle
.
馃憤 1
t
So can people on the older image based bundle upgrade to a file based one?
Or would then need to remove the old one first?
t
yes it's just yaml files that declares what the previous version was and defining channels to define what latest version is for each channel etc
馃憤 1
j
Well, shoot, I might have got greedy. First, I started to jump from just 1.6.0 --> 1.10.0, and it was looking pretty good. I notice there's a new crd, but early testing was looking good. But then I thought, I SHOULD be able to add all the in-between releases in one go as well, 1.6.1, 1.7.0, 1.8.0, 1.9.0 and then 1.10.0. But I'm wedged now. It's possible if I did it one step at a time, I'd be successful, but I'm not sure yet. Are we OK jumping 1.6.0 --> 1.10.0? Or should I see if I can get all the in between releases as well, hopefully one by one?
(I have a feeling one by one would likely work)
t
I guess it depends on if there has been any breaking changes, if so we would need to add some channels to prevent users from auto upgrading and getting broken
t
Yes, have the CRD versions changed in that time? Other than than I think going from 1.6 to 1.10 is fine in one jump, the issue is Flink version support, if they overlap then a user can upgrade to a shared version under 1.6 and then upgrade cleanly to 1.10
Looks like 1.6 was Flink 1.13 to 1.18 and 1.10 is 1.15 to 1.20
j
Well, it took me a bit to do. I decided to try to do all the releases at once, but it wasn't having it, so I ended up jumping from 1.6 to 1.10. I don't know how to test a
*flink*statesnapshots
but the flinkdeployments and sessionjobs look ok on my 4.17.9 FIPS cluster.
Copy code
oc get pods,flinkdep,sessionjob
NAME                                                         READY   STATUS    RESTARTS   AGE
pod/basic-example-d9d69498f-dfwrw                            1/1     Running   0          5m46s
pod/basic-example-taskmanager-1-1                            1/1     Running   0          5m28s
pod/basic-session-deployment-only-example-765cfdb959-pbdvs   1/1     Running   0          79s
pod/basic-session-deployment-only-example-taskmanager-1-1    1/1     Running   0          39s
pod/basic-session-deployment-only-example-taskmanager-1-2    1/1     Running   0          39s

NAME                                                                     JOB STATUS   LIFECYCLE STATE
flinkdeployment.flink.apache.org/basic-example                           RUNNING      STABLE
flinkdeployment.flink.apache.org/basic-session-deployment-only-example                STABLE

NAME                                                              JOB STATUS   LIFECYCLE STATE
flinksessionjob.flink.apache.org/basic-session-job-only-example   RUNNING      STABLE
image.png
I haven't done the operatorhubio PR yet.
t
That's great James. How are you doing the release, are you going to submit a PR to the upstream flink operator repo?
d
Hi @Thomas Cooper there is a separate operatorhub git repo it should go through to make this change. There is still the piece to put the instructions into a wiki / github and tie it to the flink operator release. Did I understand your question correctly?
t
Thanks David, I was more thinking about the generation of the bundle contents. If that is done by tweaking the configs in the
tools/olm
directory then we should probably have those changes merged upstream before we do the PR to the operator hub?
d
@Thomas Cooper I see what you are saying. @James Busche it sounds like you are doing a great job creating the Flink operator bundles at the latest levels for operatorhub and OLM. Did you use the file based approach the @Tony Garrard suggested? Tom and I are looking to get these instructions into the open - so others can do this; are we in a position to update the
tools/olm
directory with the up to date steps and scripts. Let us know how we can help.
j
Hey All, I saw there was a new release 1.11.0 - I've put them out on Operatorhubio and Red Hat. I'm not sure what it'd take to do the file based approach. Right now, to get things through, I've been doing some manual updates to the tools/olm folder to get things to work, but a short-term approach would be to do a PR to tools/olm that would eliminate the one-off manual tweaks I have to make. Long-term approach would be the file based approach. As for the instructions, I've been keeping them updated here: 1.10 Red Hat 1.10 Operatorhubio 1.11 Red Hat 1.11 Operatorhubio I'm not sure where to publicly post the instructions I've been maintaining above - do we want them in md files or in a wiki somewhere?
image.png
t
Nice James, thanks for your efforts. Unfortunately josdk/quarkus have not produced regular plugins for operatoe-sdk, so latest doesn't work with our current build and we're going to have to do some migration also
t
Thanks @James Busche, a PR to improve the upstream bundle generation would be good. I can't access the instructions you posted as they are behind IBM Auth. I think any instructions you have for generating the bundle for a new Operator version should be stored in the
tools/olm
folder, wither in the
README.md
or in a new
RELEASE.md
or similar. If you could put a PR up with those (with any IBM specific steps removed) that would let the community review it and we could work towards automating the release process as much as possible upstream.