https://flink.apache.org/ logo
Join Slack
Powered by
Hello Everyone, Anyone using flink kubernetes oper...
# random
s
Hello Everyone, Anyone using flink kubernetes operator in production? How is your experience with it? thanks
n
@Yang Wang
y
I believe apple does. cc @Gyula F贸ra We alibaba are also providing the EMR on ACK(Alibaba Cloud Container Service for Kubernetes) with flink-kubernetes-opeartor. So far so good!
馃憤 1
m
Yup, we use it at Apple. Works like a charm :)
馃憤 1
s
Great to know it works well for you. 1.Which flink operator are you using. I see there are different operators provided by apache flink community, by google and lyft. 2. Does stateful flink job upgrade work fine with operators in production? I am planning to setup continuous deployment for our flink jobs
g
There is only one operator supported by the community . https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/
It is production ready including stateful upgrades, metrics , etc
s
Awesome. I am experimenting with this currently!
g
It is newer compared to the other open source operators you have mentioned. We have taken inspiration from those to design a robust and feature rich architecture. We strongly believe this is the way forward :)
馃憤 1
s
Agreed. I was evaluating different deployment modes for flink jobs and setting up continuous deployment for our jobs. Applications mode with Flink operators looks great. Hope it meets all our production deployment needs.
g
If you are wondering about standalone/native Kubernetes integration the upcoming version will also support those options . You can already experiment with that on the main branch :)
馃憤 2
The 1.10 version uses the native Flink Kubernetes integration
s
Great! We are on flink 1.13. Planning to make use of FlinkDeployment in application mode.
馃憤 1
Will not monitoring be challenging when we start having many clusters each running only one job?
g
To get the most out of the operator 1.15 is recommended but 1.13 should also work (but not every feature)
s
Sure we can upgrade to 1.15
g
The main difference is improved error handling and robustness during failures / upgrades that 1.15 makes possible
The operator logic is generally the same for all Flink versions so all supported versions are production ready. It's only that newer Flink versions expose a few extra configs that we can leverage
s
Got it thanks. You recommend using flink version 1.15 and flink operator version 1.1.0?
g
Yes, latest and greatest :)
s
Sure thanks Gyula
g
If you are planning to build a custom operator image the release-1.1 branch contains a few fixes that will be released in a few weeks for 1.1.1
Nothing critical, just a tip
s
Thats good. Sure will check it out
g
Many users end up building their own operator image anyways due to extra metric reporters, dependencies , plug-ins etc
馃憤 1
It's highly customizable
s
Right we also plan to have support for prometheus monitoring
We have multiple flink jobs ( 5) How many flink operators are good enough to manage these 5 FlinkDeployments? Which worked well for u? one flink operator watching multiple FlinkDeployments or One operator per Flinkdeployment.
g
1 operator to watch all your deployments
you can increase the number of processing threads (default is 10 I think) but a single operator should be able to handle tens or hundreds of resources in principle
You can of course decide to have different operators in different namespaces, but there is no fundamental limit on how many FlinkDeployments one can manage
The operator can also be upgraded, reconfigured etc without affecting running jobs, so it鈥檚 pretty smooth
s
Thats good. Is it mandatory to install Flinkdeployments in the same namespace as Operators namespace?
Lets say i want to run my different flink job clusters in different namespaces
g
no, you can configure what namespaces the operator watches for resources. The operator can be in one namespace and only watch resources in others if you want
s
just to have some isolation
g
you would generally want to have the operator in an admin namespace
and jobs run in user namespaces
s
exactly
g
thats well supported
s
Should i need to configure anything explicitly?
g
check the 
watchNamespaces
setting
s
Awesome
Thanks
馃憤 1
I was able to to setup a stateful job and upgrade it seemlessly. Both stateless and savepoint types worked fine
馃コ 1
g
good news 馃檪
馃憤 1
s
i was just wondering if there would be high downtime during upgrade if parallelism is very high for a job and more TMs in a cluster
3 Views
Open in Slack
PreviousNext
Powered by