Hi, I am in process of migrating Atlantis to the new K8s version 1.24, my state file resides on S3, and I am using an IAM role to access that. But with the new deployment on K8s 1.24, I am getting: Initializing the backend... Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes. Error refreshing state: AccessDenied: Access Denied status code: 403 I have checked the IAM role, within the same pod, I am able to access the state file from S3, using AWS CLI commands, I can perform write operations on the S3 bucket. So the IAM role is perfectly fine, so is the role binding with the service account, not sure why TF isn't able to access it.

How are you specifying the IAM role, is it configured at the instance level or within your TF?

Does the backend configuration use kms? Does the role have permission to use the kms key for crypto operations on the state file(s)?

The IAM role is configured with IRSA on the pod

I would recommend checking your terraform to make sure there isn't anything override the AWS provider configuration

after trying a lot of things, upgrading the TF version made it work, we were running an old version 0.12.27, and somehow it wasn't picking the right IAM role, running TF plan within the pod from version 0.12 and 0.13, produced different results, 0.13 picked the right IAM role