jwr
05/23/2023, 1:47 PMDylan Page
05/23/2023, 2:15 PMjwr
05/23/2023, 2:16 PMross strickland
05/23/2023, 2:16 PMjwr
05/23/2023, 2:17 PMDylan Page
05/23/2023, 2:18 PMjwr
05/23/2023, 2:22 PMDylan Page
05/23/2023, 2:23 PMevents.RunAndEmitStats
seems to be the last call in the stackjwr
05/23/2023, 2:25 PMDylan Page
05/23/2023, 2:27 PMross strickland
05/23/2023, 2:30 PMDylan Page
05/23/2023, 2:34 PMafter top-level value
ross strickland
05/23/2023, 2:35 PMjwr
05/23/2023, 2:45 PMconftest
command was that i couldn't find any other way to specify the path with -p
, since the path is dynamic and changes on each run (it has the PR number or something as part of the path).ross strickland
05/23/2023, 2:48 PM-p
dynamic? are policies different for different PRs?jwr
05/23/2023, 2:55 PMconftest
command is:
/home/atlantis/.atlantis/repos/SOME_COMPANY/devops/4478/default/terraform/SOME_COMPANY/environments/THIS_ENVIRONMENT
so when we run:
conftest test -p ../../modules/aws/application/policies/
that policy path resolves to:
/home/atlantis/.atlantis/repos/SOME_COMPANY/devops/4478/default/terraform/SOME_COMPANY/modules/aws/application/policies/
The PR number in the middle of the path there is the dynamic part. The example seen on line 21 here seems to imply that we need a full path, and I wouldn't know how to make the PR number part of that full path: https://www.runatlantis.io/docs/policy-checking.html#step-2-define-the-policy-configuration
does that make sense?ross strickland
05/23/2023, 2:57 PMjwr
05/23/2023, 2:58 PMross strickland
05/23/2023, 3:11 PMjwr
05/23/2023, 3:14 PMross strickland
05/23/2023, 3:22 PMpolicies:
policy_sets:
- name: "yourpolicies"
path: ../../modules/aws/application/policies/
....
--all-namespaces
flag can be thrown on as an extra arg in the policy_check
step.jwr
05/23/2023, 3:26 PMpath
should be *absolute dir* path to conftest policy/policies
. so i may not have attempted it for that reason. i can give it a try now though...
https://www.runatlantis.io/docs/policy-checking.html#step-2-define-the-policy-configurationross strickland
05/23/2023, 3:31 PMjwr
05/23/2023, 3:44 PMpolicy_sets
suggestion by setting this variable in atlantis and recycling its container:
{
name = "ATLANTIS_REPO_CONFIG_JSON",
value = jsonencode({
policies = {
owners = {
users = [...],
},
policy_sets = [
{
name = "devops"
path = "../../modules/aws/application/policies/"
source = "local"
}
]
},
...
and then i set my atlantis.yml
workflow like this:
policy_check:
steps:
- policy_check:
extra_args: ["--all-namespaces"]
but policy checking fails with this error commented back to a PR:
1 error occurred:
* policy_set: devops: conftest: Error: running test: parse files: get file info: stat /home/atlantis/.atlantis/repos/springbuk/devops/4478/default/terraform/SOME_COMPANY/environments/production/production-default.json: no such file or directory
ross strickland
05/23/2023, 4:21 PM- show
before - policy_check
in the policy workflow section in atlantis.yaml
?policy_check:
steps:
- show
- policy_check:
extra_args: ["--all-namespaces"]
jwr
05/23/2023, 4:29 PMapply:
steps:
- init
- show # this is the equivalent of `terraform show -json $PLANFILE > $SHOWFILE`
- run: >
conftest test -p ../../modules/aws/application/policies/ --all-namespaces $SHOWFILE
- apply:
extra_args: ["-lock-timeout=300s"]
ross strickland
05/23/2023, 4:32 PMjwr
05/23/2023, 4:34 PMatlantis plan
at 11:59 and their policy check would pass. and then if they subsequently run atlantis apply
at 12:01, they should be blocked.
but i guess your suggestion is that since the apply
stage doesn't usually report policy failures back to github anyway, then we can just leave it as-is?ross strickland
05/23/2023, 4:36 PMjwr
05/23/2023, 4:44 PMross strickland
05/23/2023, 4:46 PMjwr
05/23/2023, 4:48 PMatlantis approve
, and then the apply still fails even when the maintenance window is not active, because conftest -p ...
doesn't know anything about approving anything. cool, thank you.ross strickland
05/23/2023, 4:49 PMjwr
05/23/2023, 4:53 PMross strickland
05/23/2023, 4:54 PMjwr
05/23/2023, 5:05 PMpolicy_sets = [
{
name = "devops"
path = "../../modules/aws/application/policies/"
source = "local"
},
{
name = "sftp-user-management"
path = "../../modules/aws/sftp-transfer/policies/"
source = "local"
}
]
then a PR in the devops repo will get an error saying that the sftp-user-management policies aren't found. and a PR in the sftp-user-management repo will get an error saying that the devops policies are not found.
is there a way that i can specify both policy sets in the atlantis config, and then choose which one i want in atlantis.yml
in each repo?ross strickland
05/23/2023, 5:11 PMjwr
05/23/2023, 5:13 PMpolicy_sets
list in my atlantis config. the atlantis.yml
for each repo was the only source of truth for which policies should be run in which repos, by way of conftest -p ...
pointing to the appropriate directory in each repo.ross strickland
05/23/2023, 5:18 PM-p
for the respective policy dir in each repo within the extra_args
specified in the workflow for that repojwr
05/23/2023, 5:32 PMname = "ATLANTIS_REPO_CONFIG_JSON",
value = jsonencode({
policies = {
owners = {
users = [...],
},
policy_sets = [
{
name = "devops"
path = "../../modules/aws/application/policies/"
source = "local"
},
{
name = "sftp-user-management"
path = "../../modules/aws/sftp-transfer/policies/"
source = "local"
}
]
},
this workflow in atlantis.yml
in one of the individual repos:
policy_check:
steps:
- show # this is the equivalent of `terraform show -json $PLANFILE > $SHOWFILE`
- policy_check:
extra_args:
- "-p ../../modules/aws/application/policies/"
- "--all-namespaces"
results in this error in a PR:
1 error occurred:
* policy_set: sftp-user-management: conftest: Error: running test: load: loading policies: load: 1 error occurred during loading: stat ../../modules/aws/sftp-transfer/policies/: no such file or directory
that sftp-transfer directory doesn't exist because that's meant for the other repository.ross strickland
05/23/2023, 5:41 PM-p
arg for each repo extra_args on the policy_check workflow section.jwr
05/23/2023, 5:56 PMpolicy_sets = [
{
name = "policies"
path = "/tmp" # This is a dummy path, the actual policies are specified in atlantis.yml in each repository
source = "local"
}
]
and then i can use -p
in extra_args
in the atlantis.yml
of each repo to specify the real path to the real policies.ross strickland
05/23/2023, 5:58 PMjwr
05/23/2023, 5:59 PMross strickland
05/23/2023, 6:02 PMPePe Amengual
05/23/2023, 6:14 PMRB
05/24/2023, 1:43 AMAlberto Rojas
05/24/2023, 8:39 AMinvalid character 'W' after top-level value
The issue I introduced is the one that generated the panic.
There is something else that must not be related to my metric changes since I'm not touching the policies or the json file from the plan.RB
05/24/2023, 11:48 AMAlberto Rojas
05/24/2023, 12:58 PMRB
05/24/2023, 1:04 PMAlberto Rojas
05/24/2023, 1:04 PMRunAndEmitStats
, or am I wrong?RB
05/24/2023, 1:07 PMjwr
05/24/2023, 1:09 PMRunAndEmitStats
.RB
05/24/2023, 1:09 PMAlberto Rojas
05/24/2023, 1:11 PMRB
05/24/2023, 1:14 PMAlberto Rojas
05/24/2023, 1:15 PMRB
05/24/2023, 1:19 PMAlberto Rojas
05/24/2023, 1:20 PMjwr
05/25/2023, 1:35 PMpolicy_sets
to my atlantis container config in order to get policy checking to work on 0.24:
{
name = "ATLANTIS_REPO_CONFIG_JSON",
value = jsonencode({
policies = {
owners = {
users = [...],
},
policy_sets = [
{
name = "policies"
path = "/tmp" # This is a dummy path, the actual policies are specified in atlantis.yml in each repository
source = "local"
}
]
},
my atlantis.yml
file looks like this: https://pastebin.com/StxCajkA
when we create a PR which involves terraform/environments/production
, autoplan kicks in and works as desired, including conftest policy checking.
but sometimes we have a PR which involves terraform/environments/staging
, and that directory isn't mentioned in our atlantis.yml
at all, so no autoplan happens, which is expected. in some cases we might comment on those PR's with atlantis plan -d terraform/environments/staging
and atlantis will kick off a plan. but when we do that now, after having added the policy_sets
block above, these PR's will get an error saying ``Error: running test: load: loading policies: no policies found in [/tmp]` .
is there a way to instruct atlantis to only do policy checking for directories which are defined in atlantis.yml
and do not do policy checking for directories which are not defined in that file?ross strickland
05/25/2023, 3:58 PMjwr
05/25/2023, 4:01 PMextra_args
override for -p
, which seems to cause atlantis to look for policies in /tmp
, and there are no policies in /tmp
because that was just a dummy directory.ross strickland
05/25/2023, 4:01 PMjwr
05/25/2023, 4:06 PMdefault
, and the workflow seen in my pastebin is the only workflow in the entire file.ross strickland
05/25/2023, 4:06 PMworkflows:
default:
policy_check:
steps:
run: exit 0 (or something that exits cleanly..)
jwr
05/25/2023, 4:10 PMdefault
workflow like that? or would the usual init/plan/apply steps not be overridden if i'm only explicitly overriding the policy_check
step?ross strickland
05/25/2023, 4:11 PMjwr
05/25/2023, 4:11 PMross strickland
05/25/2023, 4:11 PMjwr
05/26/2023, 6:58 PMworkflows:
default:
policy_check:
steps:
- run: /bin/true
and then when I push a commit which has no explicitly defined workflow, and then i manually run a atlantis plan -d ...
, I'm still getting the same error where it's apparently expecting policies to exist in the dummy path:
1 error occurred:
* policy_set: policies: conftest: Error: running test: load: loading policies: no policies found in [/tmp]
any thoughts on that?Alberto Rojas
05/30/2023, 1:30 PMinvalid character 'T' after top-level value
Justin S
05/30/2023, 2:03 PMAlberto Rojas
05/31/2023, 12:17 PMError running policy_check operation: unexpected end of JSON input
https://github.com/runatlantis/atlantis/issues/3438#issuecomment-1570110273