https://linen.dev logo
Join Slack
Powered by
I'd like to start a conversation about Helm chart/...
# fluent-operator
j
I'd like to start a conversation about Helm chart/release improvements -- starting with how we're handling CRDs 🧵
How we currently handle CRDs: • We currently distribute fluent-operator's CRDs in two separate "sub-charts" which are dependencies of the "main" chart. • This makes it difficult to upgrade Helm releases when there are CRD changes, because CRDs (in 
crds/
directories) are not upgraded by default in Helm. Users have to manually fetch CRDs, run 
kubectl replace
etc • The usage of 
file://
references in dependencies also causes duplicate CRDs in the release as reported here.
I'd like to propose that we implement something similar to how Karpenter handles their Helm releases: • Include the "fluentd" and "fluent-bit" CRDs in the fluent-operator's "main" chart's 
crds/
folder • This will ensure all CRDs get installed during an initial 
helm install
• Create a new "top-level" chart named 
fluent-operator-crds
• Put CRDs in the 
templates/
directory of the 
fluent-operator-crds
chart • Placing them in the 
templates/
directory will allow a user to 
helm upgrade
the 
fluent-operator-crds
chart to upgrade our CRDs instead instead of having to fetch them manually and 
kubectl replace
etc • In this setup, we'll need to publish updated CRDs to two locations -- 
charts/fluent-operator/crds
and 
charts/fluent-operator-crds/templates
-- but this should be fairly simple with CI. (edited)
I don't think this would introduce any breaking changes: • Currently, if a user wants to install fluent-operator without any CRDs, they can set 
fluentd.crdsEnable
or 
fluentbit.crdsEnable
to 
false
• Moving forward, we would deprecate these two values, and a user could simply use 
helm install --skip-crds
if they want to prevent CRD installation
👀 1
As part of this, we can also clarify and update our documentation for "installing" and "upgrading."
Any thoughts? (cc: @Marco Franssen @Chengwei Guo)
It might make more sense if I just create a new Github issue for this. I'll do that.
m
I don't think we can use the --skip-crds. It would not allow to selectively install which crds you want in the cluster. Furthermore many helm charts e.g. Spire which I also maintain use separate crd charts to give users more control. E.g. in ArgoCD I install CRDs in a separate sync wave in a separate ArgoCD application.
j
I'll respond there, thanks for the comments!
but, what I proposed does use a separate CRD chart -- I don't know if there is value in having multiple CRD charts (one for fluentd, one for fluentbit). If a user wants to only install fluentd for example, we can do this via values (eg, 
fluentd.installCrds: {true|false}
).
So, given this scenario, if a user only wanted to install fluentd CRDs, they could use 
--skip-crds
on the 
fluent-operator
chart (this can be done in Argo as well) and then 
fluentd.installCrds:true
and 
fluentbit.installCrds:false
on the 
fluent-operator-crds
chart.
added some more comments to the issue!
m
The value would be in the case for people that use an argocd app to install the CRDs, if there is no change to the chart argo cd also won't trigger a sync. using values it would still detect there is a new release for the chart, however in my case I only use fluent-bit and in that version it might only have an update to fluentd.
j
I think using an Argo app specifically for the CRDs is an interesting but not terribly common thing -- esp since Argo will try to always apply CRDs first in a sync run (even without sync waves). I would guess the majority of fluent-operator's user-base uses both fluentbit and fluentd, and asking them to install 2 separate helm charts just for CRDs (3 total) seems a little weird. The value change would trigger a sync, but it would be a no-op sync for which ever CRD group (fluentbit or fluentd) you have "disabled" -- unless I'm misunderstanding something here (which is likely!). I think either option is fine though -- two separate CRD charts, or one.
m
we also use it using a replace and server-side apply app. Furthermore we install the crds earlier so other stuff that needs the crds can already get deployed before the controller is even there. We do that with all crds so we get less sequential deployment
Open in Slack
PreviousNext
Powered by