Hello just coming back to atmos after about a year...
# atmos
c
Hello just coming back to atmos after about a year away. Amazing new additions to the tool BTW! I'm working on setting up atmos to improve the terraform reuse and organization in my current role/org and I have a question about managing the terraform s3 state key. In my previous role/org we explicitly used
workspace_key_prefix
all over our stacks to ensure state keys were what we expected. I can see now, however, there is new guidance and a
name
element in
component
metadata
. Basically I'm talking about this recommendation: https://atmos.tools/design-patterns/version-management/folder-based-versioning#recommended-use-metadataname I like this idea as it's very clean and readable in the stack config. Sorry now to my actual question. Is it true that if you rely on the default atmos behaviour of using the component (or it's new
name
) that your state bucket will be a flat listing of all your deployed components? I like to have a little hierarchy in my components to add some organization and I liked that when I set
workspace_key_prefix
this path structure can be mirrored in the state bucket. But when I don't set
workspace_key_prefix
it seems that atmos always replaces all
/
in the
component
(or
name
) with
-
. Is there any way to control this or do I need to keep specifying
workspace_key_prefix
if I want a less flat state organization?
g
@Andriy Knysh (Cloud Posse)
a
@Charles Smith your comment is correct
Copy code
I think this is the most relevant section of the docs for you as it defines how atmos decides that the workspace_key_prefix is derived which then decides where the state is in the bucket:
<https://atmos.tools/design-patterns/version-management/folder-based-versioning#how-atmos-calculates-workspace_key_prefix>
If you actually set workspace_key_prefix to something static in your top level backend.s3 then it would actually be the same state file for all (which would not be good).
But based on these docs if you do not define workspace_key_prefix then it should default to the component name (and then the actual workspace is a dir under that).
if you don't (statically) set
workspace_key_prefix
, Atmos calculates it, and in the state bucket you'll see the
workspace_key_prefixes
folders with the
workspace
subfolders (which in almost all caes are the stack names)
c
Right and it's not possible to configure atmos to skip replacing
/
in component names with
-
when it determines the
workspace_key_prefix
for you? I would like to organize my components in directories and then reference them like
services/consul
in my stack config. But it seems like the state bucket will always be a flat listing of all components as atmos turns this into
services-consul/workspace-name/statefile
.
Or I can use:
Copy code
terraform:
  backend:
    s3:
      workspace_key_prefix: "{{ .component }}"
and I do get
services/consul/workspace-name/statefile
but then I can't use the new
name
metadata or cleaner way.
a
if we don't do this (replace
/
with
-
in the component names), then the folder/subfolder structure would be completely different
which looks like is what you asking for
services/consul/workspace-name/statefile
let's reset 🙂 do you want ``services/consul/workspace-name/statefile` or
services-consul/workspace-name/statefile
?
c
Yeah we have the potential for well over 600 components so I would prefer if they had some hierarchy in the state bucket. I'm looking for the former because then it will align with my component dir.
a
ok, so this
services/consul/workspace-name/statefile
and you are saying that you can achieve it by using
Copy code
terraform:
  backend:
    s3:
      workspace_key_prefix: "{{ .component }}"
but not
metadata.name
?
c
Yes that's correct.
a
then it's an omission, we should review it and allow users to specify whatever they want in
metadata.name
and don't change it thanks for reporting, we'll take a look
c
Just to clarify I was following that doc and wanting to omit the
workspace_key_prefix
line in the backend config so atmos would generate it based on
component
or
metadata.name
. When I do that then the generated prefix has no
/
in it. Yeah ok
a
yes, this is the current behaviour (which Atmos had for a long time)
Copy code
based on component or metadata.name . When I do that then the generated prefix has no / in it.
c
Yeah I also understand that. I was just wondering if there was a flag to control the char replacement or something I was missing
a
i think we need to add a setting in
atmos.yaml
for you to override the behavior we don't want to just change it b/c it will break a lot of exsiting configs
👍 1
c
of course. makes sense