https://cloudposse.com logo
Join Slack
Powered by
hey guys, is anyone using `!store` with `auth`? I ...
# atmos
d
hey guys, is anyone using 
!store
with 
auth
? I opened an issue, but I believe there are some bugs. Probably needs the same treatment as the 
!terraform.state
functions
Finally getting around to atmosifying our infra! But I want to use store so that I can cleanly pass values from atmos + terraform to typescript + CDK via SSM
e
I believe we recently adddd support for identities with stores
I forget who it was that reported it
Cc @Andriy Knysh (Cloud Posse)
d
could you share an example of how to set an identity for store? I was using this post from February for reference for example I had this:
Copy code
# atmos.yaml
stores:
  ssm/core-root:
    type: aws-ssm-parameter-store
    identity: core-root
    options:
      region: us-east-1
      prefix: /atmos/
  ssm/tembo-dev:
    type: aws-ssm-parameter-store
    identity: tembo-dev
    options:
      region: us-east-1
      prefix: /atmos/
then in stacks, select the store:
Copy code
# stacks/orgs/aws/tembo/dev/_defaults.yaml
terraform:
  hooks:
    store-outputs:
      name: ssm/tembo-dev
and to pull from a different account:
Copy code
vars:
  output_key: !store ssm/<account> <source-component> <output-key>
a
Atmos 1.218.0 has this PR merged https://github.com/cloudposse/atmos/issues/2369
@Dan Miller (Cloud Posse) it might solve the issues that you reported. If not, we'll look into the issue
could you share an example of how to set an identity for store? I was using this post from February for reference
is the example working for you (besides the issues you reported)? Not sure if you are asking because you think the example is not correct, or it's not working b/c of the issues
was added in https://github.com/cloudposse/atmos/pull/2099 (months ago, please let me know if we need to look at the issue you reported)
d
Unfortunately I don't think it's working on 1.218.0. There are a few issues. They're likely related, but here's what I've found: 1. Non-generic stores aren't triggered
Copy code
hooks:
    store-outputs:
      command: store
      events:
        - after-terraform-apply
      name: ssm/tembo-dev
      outputs:
        vpc_flow_logs_bucket_arn: .vpc_flow_logs_bucket_arn
        vpc_flow_logs_bucket_id: .vpc_flow_logs_bucket_id
Store config:
Copy code
stores:
    ssm/tembo-dev:
      type: aws-ssm-parameter-store
      identity: tembo-dev
      options:
        region: us-east-1
        prefix: /atmos/
Trace shows the hook is completely silent — no "Fetching" output, no errors. atmos describe component resolves the hooks correctly but they never execute:
Copy code
DEBU  Running CI hooks event=after.terraform.apply force_ci=false
  DEBU  CI provider not detected provider=generic
  DEBU  CI platform not detected, skipping CI hooks error="CI provider not detected"
  DEBU  Telemetry event enqueued
2. Generic stores make it further, but are caught by the CI check
Copy code
hooks:
    store-outputs:
      command: store
      events:
        - after-terraform-apply
      name: ssm  # <---- generic store, no identity
      outputs:
        vpc_flow_logs_bucket_arn: .vpc_flow_logs_bucket_arn
        vpc_flow_logs_bucket_id: .vpc_flow_logs_bucket_id
Same trace — CI hooks intercept the event and skip because no CI provider is detected locally. Store hooks never fire. 3. When CI is disabled, generic stores make it further, but then fail on auth With 
ci.enabled: false
, the hook fires but can't read terraform outputs:
Copy code
atmos terraform apply vpc-flow-logs-bucket -s tembo-use1-dev -- -auto-approve
...
  Fetching vpc_flow_logs_bucket_arn output from vpc-flow-logs-bucket in tembo-use1-dev
  ✗ Fetching vpc_flow_logs_bucket_arn output from vpc-flow-logs-bucket in tembo-use1-dev
Copy code
Error: failed to retrieve terraform outputs: hook "ssm" (event "after.terraform.apply")
  failed to get terraform output "vpc_flow_logs_bucket_arn" for component
  "vpc-flow-logs-bucket" in stack "tembo-use1-dev":
  failed to retrieve terraform outputs: exit status 1
  Error: No valid credential sources found
4. Finally, with CI disabled, generic stores, and running with predefined auth, it works
Copy code
atmos auth exec --identity tembo-dev -- atmos terraform apply vpc-flow-logs-bucket -s tembo-use1-dev -- -auto-approve
...
  ✓ Fetching vpc_flow_logs_bucket_arn output from vpc-flow-logs-bucket in tembo-use1-dev
  ✓ Fetching vpc_flow_logs_bucket_id output from vpc-flow-logs-bucket in tembo-use1-dev
I'm checking now if it's related to profiles. I'm disabling profiles to see if that works
seems to still have issues. I'm not sure what the deal is, so I'm just swapping to 
!terraform.state
a
thanks for the info, please describe all the issues you see (if they are not in the gh issue yet), and we'll fix it asap
d
no worries! sorry for the dump of information. I'll attach some claude dump to see if you can pass it along to your agents to make anything of it
debug.txt
debug.txt
a
thanks, it's a good info
m
This might be an issue with hooks then because I use Atmos Stores with identity since 1.217.0 and had no issue both local and with CI
@Dan Miller (Cloud Posse) have you try with version 1.217.0 just to be sure it's not a new change?
d
yes I tried with 1.217.0 but the same issue
but I figured out some more details
if I pass 
-s mystackname
, the hook fires correctly if I dont pass 
-s
and then instead select the stack with the choice prompt, the hook doesnt fire!
however, even if the hook fires, the identity piece fails because the role used to access terraform state isnt correct. ie for the hook to read tfstate, we need to do dev role -> tfstate access role but the identity is just the dev role. the hook doesnt attempt to assume the backend access role
Copy code
stores:
  ssm:
    type: aws-ssm-parameter-store
    identity: dev
    options:
      region: us-east-1
      prefix: /atmos/
identity dev is correctly assumed, but identity dev cant access tfstate and therefore fails on "failed to retrieve terraform outputs"
furthermore, if I just remove the identity option altogether
Copy code
stores:
  ssm:
    type: aws-ssm-parameter-store
    options:
      region: us-east-1
      prefix: /atmos/
then start a shell in the account I want to use for store:
Copy code
atmos auth shell --identity dev
then run the command (WITH 
-s stack
)
Copy code
atmos terraform apply vpc-flow-logs-bucket -s dev
it works!
a
Dan, thanks for the additional info
👍 1
d
I'm going to close my current github issue and create 2 more that are more accurate for the actual issue
👍 1
2 Views
Open in Slack
PreviousNext
Powered by