https://cloudposse.com logo
Join Slack
Powered by
Hi guys! I have a question how do you guys manage ...
# atmos
m
Hi guys! I have a question how do you guys manage multiple orgs with the same structure/stacks do you just copy/paste the atmos to create a new repo per org? or change the default stack naming to just add the org as a prefix? I was thinking that maybe having a switch between org can be cool but maybe this is just out of scope.
a
good questions, and you already described the two possible solutions: ā€¢ A separate repo per org (in this case, the stack name pattern remians the same) ā€¢ Multiple orgs in the same monorepo
for multiple-orgs in one monorepo, in the 
orgs
folder, you would have: orgs: acme1 ..... acme2 ....
then in 
atmos.yaml
, you use this stack name template:
Copy code
name_template: "{{ .vars.namespace }}-{{ .vars.tenant }}-{{ .vars.environment }}-{{ .vars.stage }}"
each org 
_default.yaml
would have its own 
namespace
then, when you provision, you have the namespace as the first part of the stack name
Copy code
atmos terraform apply <component> -s acme1-plat-use1-dev
you can add many orgs
e
I would say there are a couple patterns actually, and I think one of these design patterns we haven't documented but maybe should.
šŸ‘ 1
So as Andre's alluding to, one option is you have an orgs folder and under the orgs folder you have a folder for each organization. That's what we primarily recommend.
a
e
This is if you want to reuse a lot of the infra across AWS Organizations. And we've seen this done where you literally will have a production AWS Organization and a dev Organization. This is great when you want to be able to allow engineers to experiment at an organizational level without cowboy coding in production.
The other use case is of course you have another AWS organization such as through acquisitions M&A. Maybe it makes sense that they are separate git repos if they're not going to share too much.
m
yeah and those for the most part work really well since most of the time you will probably create a new Org and have different components/stacks
but in my rare case that I have right now, instead of that I have to maintain the same infrastructure across Orgs so making a new repo might work but is going to be a copy/paste place for the catalogs since I can just move the components to a different repository and pull from there
but the structure of it will mostly be the same for the exemption of maybe the root accounts on both of them
maybe adding the 
org
name as prefix will work but this will be a bunch of refactoring the tf state so either way a big refactor is coming hahaha
a
no need for a big refactoring
šŸ‘€ 1
you can maintain the existing components in the old repo, and use the new pattern for the new org
give me a minute
let's say you have an old 
acme1
org, and the stack name pattern for it was 
tenant-environment-stage
in 
_defaults.yaml
for 
acme1
org, you have
Copy code
terraform:
  vars:
   label_order:
    - tenant
    - environment
    - stage
    - name
    - attributes
    descriptor_formats:
      account_name:
        format: "%v-%v"
        labels:
        - tenant
        - stage
      stack:
        format: "%v-%v-%v"
        labels:
        - tenant
        - environment
        - stage
now, you add another org 
acme2
to 
orgs
in 
_defaults.yaml
for 
acme2
, you would use
Copy code
erraform:
  vars:
   label_order:
    - namespace
    - tenant
    - environment
    - stage
    - name
    - attributes
    descriptor_formats:
      account_name:
        format: "%v-%v"
        labels:
        - tenant
        - stage
      stack:
        format: "%v-%v-%v-%v"
        labels:
        - namespace
        - tenant
        - environment
        - stage
in 
atmos.yaml
, you would change
Copy code
name_template: "{{ .vars.namespace }}-{{ .vars.tenant }}-{{ .vars.environment }}-{{ .vars.stage }}"
since all components in the old 
acme1
will not have the 
namespace
in their names, then the resources would remain the same the stack names change, so even for the old org you would use 
acme1-plat-use1-prod
i might missed something, but you can try it
@Gabriela Campana (Cloud Posse) please crete a task to document all this info in Atmos Design Patterns
šŸ‘€ 1
m
Interesting I will try this but I want to understand it a bit more, dose the 
label_order
and the 
descriptor_formats
changes how 
name_template
works? so for 
acme1
it removed the now default 
namespace
prefix from the 
name_template
for this org? but keeps it for the rest in this case just 
acme2
?
a
the idea is: ā€¢ add the 
namespace
to the stack names (to be able to distinguish b/w acme1 and acme2 stacks) ā€¢ for the old 
acme1
org, keep the same 
label_order
and the 
descriptor_formats
so the names/IDs of the resources remain the same. In this case, you still use stack names with the namespace (e.g. 
acme1-plat-use1-dev
, but the namesapce is 'virtual' to sutttisfy the atmos stack name template ā€¢ for the new 
acme2
org, use 
namespace
in the resource names (so they don't collide with the already provisioned resources in 
acme1
if you go this route, the resource names/IDs in the two Orgs will not be symmetric (the new ones will have the namespace prefix, but the old ones will not), but it should allow you to not refactor all the existing components
if you want to try it and have questions, let us know
m
I see now this sounds awesome let me try it and I will give my feedback in a bit
a
(we'll also need to correctly configure terraform workspace and workspace key prefixes for the old components so TF would see the old/existing workspaces))
m
I think I do have the namespace there so maybe I should be good
well I was wrong as you predicted terraform workspaces key prefix is an issue
old org changes the workspaces key prefix and adds the namespace so it dose not see the old state
hmm this approach aslo messes up with the 
!terraform.state
calls so refactor is need it to add the 
namespace
when calling other stacks for example
a
use the template in the old org
m
is 
namespace
not avaailable when calling 
!terraform.state
?
a
!terraform.state
uses the same stack name template from 
atmos.yaml
m
yeah I did change that but I did it on the 
_defaults.yaml
even do it mention that you can only do it on stack level hehe
a
if you don't specify a stack, then it's the same (current) stack
you can also use templates:
you can use templates (possibly with logic) while it's in transition
m
hmm I hard code it for now just so I can test the workflow change
it did not work updating the org _default with that key prefix
I just put it on the wrong place but it looks like it half works, it's still change the key on the s3 bucket to add the namespace as prefix so I just need to change that as well and we are good
šŸ‘ 1
g
m
@Andriy Knysh (Cloud Posse) is there a way to override the terraform_workspace_patter in the org 
_defaults.yaml
? https://atmos.tools/components/terraform/workspaces/
@Andriy Knysh (Cloud Posse) it looks like this will be easily solve if I can just globally manage the 
terraform_workspace_pattern
and update the 
name_pattern
to have the 
namespace
since the 
terraform_workspace_pattern
manage also the name to look for in the terraform backend this basically solves all our previous attempts do this with labels. and actually labels changes have an unintentional impacts when naming components use namespace
a
terraform_workspace_pattern
is under 
metadata
which is currently per component - you can define it per component, but I understand it's a lot of configuration (on the other hand, 
workspace_key_prefix
is under 
backend
so it can be defined globally) we have a task to make it globally configurable, we can impelement it and release in the next release (prob next week)
m
oh that awesome! I can definitely wait then using that this will be much simpler
šŸ‘ 1
Open in Slack
PreviousNext
Powered by