Title
b
Bruno Schaatsbergen
05/16/2023, 8:15 PMHas anyone tried tackling scope creep of IAM bindings added to the Atlantis service account using things like service account impersonation per Google Cloud Project to prevent Atlantis from getting 'godmode' organization wide?
Many of our clients have expressed concerns regarding the extensive access granted to Atlantis, as the service account (role in AWS terms) associated with Atlantis has privileges spanning multiple GCP projects
You could define workflows on server side level, where you force service account impersonation per workflow - but this means you have to redeploy Atlantis every time you make changes to that file...
It would be really really nice if Atlantis could make use of Google Cloud Workload Identity Federation to dynamically inject credentials per workflow, so that you can pass a service account (role in AWS terms) per workflow.
p
PePe Amengual
05/16/2023, 8:23 PMwe we did in aws is to use a federated role from a cicd account
so that atlantis role can be nucked and all the applies will fail
and same from the destination accounts
but that is not dynamic
b
Bruno Schaatsbergen
05/16/2023, 8:25 PMdid you configure an assume role in the AWS provider? and the role attached to Atlantis was allowed to assume that role?
If so, I still want to prevent that any engineer could still just put in any other role that Atlantis has access to assume
What I'm looking for is that I want to force particular repositories to assume/impersonate a role/service account
and that they can't bypass that
p
PePe Amengual
05/16/2023, 8:28 PMyes atlantis assume a role in the cicd account then that role assumes roles in the destination account
so in our case we have this :
provider "aws" {
region = var.region
assume_role {
role_arn = format("arn:aws:iam::%s:role/cicd-%s-atlantis-role", var.account_number, var.account_name)
}
}so someone could hardcode that but they will need know the name of a working role etc….
b
Bruno Schaatsbergen
05/16/2023, 8:56 PMHmm yes, I could achieve this too by utilising a similar feature in the GCP provider block, but would like to prevent teams from still having the ability to be able to assume any other possible role. Many clients roll out Atlantis for data engineering, dev teams to be able to make use of Terraform too (as what's great about Atlantis is that it makes it so easy to onboard such teams). Unfortunately large clients that have strict IAM security/governance policies, seem to struggle with adopting Atlantis for this exactly this reason: the Atlantis identity has possible god-mode access
One way of going about this, is defining all the workflows on the server side config, so that it's set in stone which workflow must use which service account/role - but the downside is that you have to redeploy Atlantis after each change.
p
PePe Amengual
05/16/2023, 9:16 PMthe workflow route is the only way so far
but I wonder if using something like Hashicorp Vault you could achieve this, but then that is yet another thing to manage
OIDC support will be great , that way you get temporary credentials
b
Bruno Schaatsbergen
05/16/2023, 9:21 PMOIDC would be very nice
p
PePe Amengual
05/16/2023, 9:22 PMPRs are welcome
b
Bruno Schaatsbergen
05/16/2023, 9:22 PMHappy to look into this actually, will put a proposal together
✅ 1
Thanks for the quick brainstorm
c
Chastity Blackwell
05/17/2023, 1:49 PMYeah, when I was designing our implementation I was trying to figure out how do this...but the solution I used was just to use one Atlantis host for our nonprod folder and one for prod. Everything else seemed like it could potentially be subverted. We were keeping all our infra code in a single repo, though, so...
m
Marcus Ramberg
05/17/2023, 1:54 PMWe’re running many different atlantis instances with different SAs to avoid having a god account with too much privs
(We run the instances in different namespaces in GKE and use workload identity to provision the individual SAs).
b
Bruno Schaatsbergen
05/17/2023, 4:47 PMHmm right, does this mean you have split it up per 'domain/department' and you have various subdomains?
e.g.
• data.atlantis.company-x.com
• platform-team.atlantis.company-x.com
• network-team.atlantis.company-x.com
@Marcus Ramberg
Right on, @Chastity Blackwell with that setup you are good. It just becomes a pain when having more teams that all roll out Terraform individually
m
Marcus Ramberg
05/17/2023, 4:48 PM@Bruno Schaatsbergen yepp
✅ 1
b
Bruno Schaatsbergen
05/17/2023, 4:49 PMI'll see if I can introduce OIDC to Atlantis, so that credentials are dynamically injected when a workflow is kicked-off. Still need to wrap my head around how we should design this.
But a similar implementation to: https://github.com/google-github-actions/auth would do the job I think (Workload Identity Federation) + a OIDC token provider running in Atlantis.
What are your thoughts on this @Marcus Ramberg?
m
Marcus Ramberg
05/17/2023, 4:55 PMYou’re thinking for running a single instance and assuming a SA dynamically at job start? Sounds interesting, I’m not really sure how it can be designed to ensure other repos are limited from assuming each others service accounts tho.
✅ 1
b
Bruno Schaatsbergen
05/17/2023, 4:57 PMUsing Workload Identity Federation we can configure a pool per project, including a SA per project. The SA would only be 'owner/editor' on that specific project. Atlantis would have access to impersonate that SA. With this setup you would not prevent other repos from assuming each others service accounts, but you would reduce the internal blast radius heavily I think
m
Marcus Ramberg
05/17/2023, 5:03 PMwe’re currently running 14 instances with a templated helmfile to manage the various variants. it’s not really that much overhead to run.
b
Bruno Schaatsbergen
05/31/2023, 5:38 PMI'm making a head start on adding Workload Identity Federation to a 'project'
👍🏽 1
Draft PR if you are keen to look a long, just started: https://github.com/runatlantis/atlantis/pull/3467/files. There will likely be a few mistakes as it's my first larger contribution to the codebase.
p
PePe Amengual
06/01/2023, 3:25 PMquestion, why it has to be GCP specific? can this be made so it works with any provider?
b
Bruno Schaatsbergen
06/05/2023, 10:50 AMI think that the overall OpenID implementation should definitely be made specific yes
But each cloud provider has a different way of authenticating using OpenID
AWS uses IAM roles and GCP uses Service Accounts
But from an implementation perspective we should make it generic, and I suppose that using either a project or workflow, we can inject an aws iam role or an gcp service account. I still have to properly think this through.
p
PePe Amengual
06/05/2023, 3:35 PMawesome , thanks Bruno