Hey, everyone. Random unimportant question. I thi...
# atmos
n
Hey, everyone. Random unimportant question. I think I know the answer, considering the obvious focus of Atmos and Atmos Pro using Github/Gitlab. But I'm curious. Has the team ever considered a Kubernetes Operator approach similar to tofu-controller or burrito? We've been using Atmos for some time now, but we're also a heavy Kubernetes + ArgoCD shop. So, naturally, I found the approach intriguing.
e
It's not out of the question and obviously we use a lot of Kubernetes as well over the years
One move in that direction though is that we have added Git enablement to Atmos
What that means is Atmos can now clone repositories and commit to them as part of life cycle hooks
We're coming out with native support for Kubernetes components (also supporting kustomize), as well as adding support natively for Helm. This means that Atmos will be able to render manifests and also commit them directly to a deployment repository.
So this is handling part of the story, just not the part of the story that happens inside the cluster
n
Interesting. Your answer only reinforces what I already suspected: that you guys could probably crush the admittedly niche "terraform in K8s" space. I've been experimenting with tofu-controller, and I find it really opens up a ton of unexpected potential. But it lacks polish at the moment, not to mention the many other features tools like Atmos provide. Anyway, I can't say we'll use the Kustomize/Helm features much, but I'm glad to hear they're on the way. We also use Atmos for some local development setups (minikube), so it will certainly be welcome there. I'm looking forward to see how Atmos interacts with Kustomize (we also happen to use a lot of Kustomize over Helm).
e
How do you bootstrap ArgoCD or things not managed via k8s gitops?
n
Well, after giving it more thought, I suspect we would possibly use an Atmos Kustomize/Helm feature. We bootstrap ArgoCD by using the Terraform Helm provider to deploy ArgoCD. This is meant to be a one-time action; the lifecycle ignores all changes afterwards. Then we deploy a couple of parent apps (app-of-apps) using the Terraform ArgoCD provider. One of those parent apps takes over managing ArgoCD itself, which is why the helm chart resource mentioned earlier ignores all changes. But it would be nice if Atmos could just deploy the same raw YAML manifests that the parent app eventually uses, anyway, to bootstrap ArgoCD.
e
Yes I think you would benefit from this native implementation of deploying rendered Helm manifests. And it supports a diff/plan preview.
I am curious about the operator approach. Can't say I've spent much time thinking about it so I don't have a mental model of how it could or would work. It's also one of the reasons we've stayed away from things like cross-plane. I think it would be really interesting to talk with you if you've spent any time now working with Burrito or Tofu Controller.
n
Oh, well, I don't see why not. I'd be happy to talk you or anyone about it at length if you're interested. I'm not an expert. But I think I've spent nearly a month actively experimenting with Crossplane, ACK, and Tofu Controller. I have ACK + Tofu Controller working for a development environment now. I originally built it with Crossplane, but decided against it in the end. I also avoided Burrito because it lacks a concept of dependencies between what they call "Terraform Layers" (something akin to a single component in a specific Atmos stack). It's a shame, too, because I think their companion UI is rather nice. My overarching thoughts are • Crossplane ◦ The approach is extremely powerful and since it runs in a K8s cluster, the potential for integration with other tools/processes/workflows is very alluring. However, it took me a week just to understand it enough to use it and training others sounds awful. There's no good way to preview changes/diffs, which seems to be a "design decision" that I would probably categorize as something more like "major flaw". • ACK + Kro ◦ Extremely powerful once again. The YAML + CEL syntax is awkward at first but quite robust, if a tad verbose. It's also far simpler than Crossplane, which means the lack of a change preview is less alarming. The automatic config drift reconciliation loop is awesome. However, I still want to preview changes sometimes. You can sort of suspend reconciliation, but it's not quite the same. The ACK CRD quality is also a bit inconsistent, which is probably where Crossplane is better. • Tofu Controller ◦ In theory, this is all of the greatness of Crossplane and ACK but with far less of the bad. You can use existing Terraform code, you can get all of the nifty perks of an operator reconciliation loop, you can take advantage of all of the integration possibilities in K8s, and you can toggle 'auto apply' on or off to preview changes when its appropriate. Even the PR workflow is elegant, in theory. However, there are some design decisions that concern me. I think they could easily result in an accidental 'auto apply' unintentionally. The project is also a bit rough around the edges, especially the companion CLI.
In the end, my conclusion goes something like... Tofu Controller is nearly there and might actually beat Crossplane and ACK at their own game. But I'm not sure it's production-ready and I'm pretty sure an "Atmos Controller" could do it better.