https://cloudposse.com logo
Join Slack
Powered by
Hi all. I am running through the tutorial for atmo...
# geodesic
j
Hi all. I am running through the tutorial for atmos to understand how to port our old multi-account structure the new workflow. Am I correct in understanding all of the state files will now go into the root account?
p
yes, that is how we use it
on bucket with subfolders basically
j
Thanks @PePe Amengual. Right, that's what I inferred from running through the tutorial. Just wanted to make sure that is the intended setup.
Does this path look correct to you, for a component state file in an account / env? --
Seems 
env:
is incorrect?
p
mmmmm
j
I'm actually on my way out, so I will look into it later. But it seemed like a missing interpolation for some var.
p
yes I think you are missing something
we use : "workspace_key_prefix"
j
ok, i ran the stock tutorial. I found one issue, so perhaps this is another minor one. I will put a PR up once i try that change.
👍🏽 1
e
@Igor Rodionov has written a script to help migrate it for one of our customers
j
Is that something you could share @Erik Osterman (Cloud Posse) ?
i
@Joe Hosteny Hello. I can share the script on monday - I have to remove project specific settings and edge cases
j
Thanks @Igor Rodionov. That would be great. I am working on trying to port one of my orgs (built with reference arch) to the new structure, starting with the 
account
module. I am taking notes, and am happy to contribute those back as well if it is helpful to others.
m
I ran into the above issue as well (with "env:" leaking into the s3 bucket path) -- defining workspace_key_prefix, and making sure there was an 'environment' and 'stage' set in the top level yaml files seemed to do the trick.
fwiw, I also have a bunch of tutorial notes which may be helpful to some -- not sure where I should upload them, however, so they do not get in the way.
j
@marc slayton thanks for your help yesterday. I have some notes too - happy to set up a shared repo if you would like to record those somewhere in the interim, and I can add mine as well
m
I have an example set of configs that got me to a successful multi-account spin up the first time (beginner's luck, I expect). I am hoping to finish a procedure which works reproducibly, even if the initial tfstate-backend gets overwritten. This is a common gotcha, I think -- an easy mistake to make in the initial config.
On my second multi-account spin up, I made the mistake of using the same bucket and workspace definitions for the tfstate-backend initialization and for the root account spin up. This had the effect of overwriting my initial tfstate and left me needing to reimport the root account.
I think the problem can be corrected by importing the root account definition -- but as you know, Atmos has a bug in the terraform-core.variant file, which is blocking me from testing the workaround.
It's a little embarrassing, but learning how to recompile atmos has taken longer than I thought. Not much of a roadmap there just yet, and there are lots of unfamiliar tools (for me anyway), and fiddly details. As soon as I have a fresh compile of Atmos, I will forward you the procedure that worked, and also the workaround for the gotcha. If I can't figure that out today, then I will forward what I have and cross my fingers that it works for you.
j
Hi Marc, I was able to do this with the Dockerfile in the 
examples
directory in the 
atmos
repo. I then just forked the repo for the module changes and pointed it to my branch with the fix for the region issue. Let me see if there are any other changes from that.
Copy code
diff --git a/atmos/modules/terraform/terraform-core.variant b/atmos/modules/terraform/terraform-core.variant
index e0f0556..fdb9321 100644
--- a/atmos/modules/terraform/terraform-core.variant
+++ b/atmos/modules/terraform/terraform-core.variant
@@ -220,8 +220,7 @@ job "terraform provider override" {
       <<-EOS
       provider:
         aws:
-          - region: "${opt.region}"
-            assume_role:
+          - assume_role:
               role_arn: "${opt.role}"
       EOS
     ,
atmos.docker.diff
atmos.docker.diff
atmos.variant.patch
atmos.variant.patch
The first patch is (obviously) my copy of the dockerfile. Second is 
main.variant
in the 
cli
subdir. You can import with these changes.
The 
make docker/build
will export a binary with the new options, so 
atmos
will have the fix when run in the built container
The 
tfstate-backend
component also has bucket versioning enabled. It saved me once when I manually deleted the wrong workspace's statefile going through this.
m
Hey Joe -- all the above makes sense to me, but there is still something obvious I seem to be missing. I have a similar repository to the one you rmention, only mine is called 'gangofnuns'. I use the same name for my forked repository on github: https://github.com/gangofnuns/atmos/releases/tag/v0.17.2-beta.1 I can build a binary and upload it, given a few extra variables on the command line, however, when I use Geodesic to pull down my version of the docker image, it contains no atmos at all. So clearly, I have the flow incorrect in some way. This is a good exercise though, as I'm sure this is something other people are wondering about. It's complicated, and there's no roadmap -- I'll keep digging this afternoon/eve.
Problem appears to be here, when I run the build (NOTE: had to add a variable to get it working with the docker registry.)
Copy code
DOCKER_IMAGE_NAME=gangofnuns/atmos make docker/build
Copy code
=> ERROR [5/8] RUN apk add atmos@gangofnuns                                                                                                                                   1.1s
------
 > [5/8] RUN apk add atmos@gangofnuns:
#9 0.983 WARNING: The repository tag for world dependency 'atmos@gangofnuns' does not exist
#9 0.983 ERROR: Not committing changes due to missing repository tags. Use --force-broken-world to override.
------
executor failed running [/bin/sh -c apk add atmos@gangofnuns]: exit code: 99
make: *** [docker/build] Error 1
j
What do you mean by "use Geodesic to pull down my version of the docker image"?
m
The flow is a little complicated, but here's how I understand it right now:
1.) pull down the github repository for cloudposse/atmos.
2.) make the code changes we talked about.
3.) Create a new docker container with the code changes. Upload to a different docker registry.
4.) Change the reference in the Geodesic container build to pull in the new container you just uploaded to the registry.
5.) Run 'make all' from within a repo such as the 'multi account reference' repo.
With an edit to the Dockerfile, this should pull down the atmos container I built above, and use it to create the Geodesic container.
Even as I write this, it seems way too complicated. I must be missing something obvious about the intended flow.
j
I'm a bit confused by what your new docker container is, vs. the reference in the "Geodesic container build". I am only using one container - the one in the example repo. The line 
RUN variant2 export binary $PWD $CLI_NAME
creates 
atmos
, or whatever you call it if you change that name. So I do a 
make docker/build
, 
sudo make install
, and then run 
<http://atmos.mydomain.com|atmos.mydomain.com>
, and if you do 
which atmos
when that runs, you will see one in /usr/local/bin/. 
CLI_NAME
in the dockerfile controls the exe name, whereas the installable container name is set by 
APP_NAME
in the Makefile.
That is, I set 
APP_NAME
in the Makefile to 
atmos.<somedomain>.com
, since I will have multiple versions of the geodesic container, one for each set of accounts managed (since we have multiple segregated projects)
So, from my bash command line, I run 
<http://atmos.crl.com|atmos.crl.com>
, which starts the container built from that Dockerfile, and then 
atmos
from the command prompt inside that running container
It sounds like you might be using two dockerfiles, and having one based on another? If so, is it possible you didn't copy the binary to the latter stage?
I've got to run for the evening, but if you are around tomorrow, send me a PM and perhaps we can do a quick zoom?
m
RE: Geodesic container build -- I've been following the instructions for building a multi-account stack. The instructions say: "Go get this repository, with these specific components, and stack definitions". That repsitory comes with a Makefile and a Dockerfile for building Geodesic. That's the container I'm using, and where I've been assuming I'd need to install the newly recompiled atmos.
Yes, I will have worked through my confusion by then. (So embarrassed.) And in any case, I will send you what I have in terms of procedures for multi-account spin up. RE: Zoom - have you tried Gather Town instead? If not, no problem -- I can do Zoom too. The password for the Gather space in the link is "so-foo-bar". Works in any Chrome browser. Cheers --
j
Thanks, that call cleared up what you are doing
m
Appreciate the extra time, Joe -- many thanks. 🙏
Sorry I didn't get to show you Gather Town, but maybe some other day. It is a fun step beyond Zoom, imho. Will send some preliminary spin up notes this eve. I am in California, btw, so it might be late. Cheers --
@Joe Hosteny - here's my original template for a multi-account run. Notice that initially, I set the tfstate-backend to local, not s3. I later copied it to S3 in the master account. after accounts were spun up. I have more procedural notes, but it's a bit late. I'll have to finish organizing those for tomorrow. Enjoy!
master.yaml
Hey Joe -- here's what I have thus far from a notes perspective. Hope to add more this evening after some other weekly chores. Still very much a work in progress. Cheers --
👍 2
e
cc: @Andriy Knysh (Cloud Posse)
✅ 1
m
^^^ quick caveat -- these are embarrassingly incomplete. Slowly trying to improve. Very sorry for delays.
Quick update: With a change to terraform-core.variant and a rebuild of atmos, I was able to import the master account, and thus get my second multi-account reference architecture up and running, despite having botched the tfstate-backend setup. So now I have two procedures that seem to work. This is using terraform 0.14.10.
Important note: I ran into this bug after importing my master account. The workaround was to add:
Copy code
# There is no AWS Organizations API for reading role_name
  lifecycle {
    ignore_changes = ["iam_user_access_to_billing"]
  }
to
Copy code
components/terraform/accounts/main.tf
seems to have worked without issue. I've voted up the bug fix. Hopefully that one gets looked at soon.
Still recording my jolly antics here.
👍 2
j
Hi @marc slayton - made a lot of progress over the weekend, including finishing deploying core iam / dns / cloudtrail across all accounts, and importing a number of pre-existing resources created via the old ref arch. Happy to sync at some point if it is helpful.
m
I would most definitely be interested. I'm looking for sort of a 'hit list' of things I would need to address while back porting existing modules.
With such a list, or a condensed series of liner notes, I could probably make decent progress on backporting further modules.
Open in Slack
PreviousNext
Powered by