Using Atmos 1.220, one final change would be aweso...
# atmos
a
Using Atmos 1.220, one final change would be awesome to achieve what I'm at properly: Use local components for dev environments, while pinning these same components for prod environments. This allows to safely change component code in PR with impact only on dev, and restrict what affects prod to only the prod stack files and a prod catalog (see below) which can be protected by CODEOWNERS. This seems to be the only way to have dev and prod in the same repo while still allowing to vet dev changes with a usual GitHub PR approval process, while ensuring such changes can't change prod without a special approval enforced by the CODEOWNERS. So, for each usual stacks/catalog/component/XXX.yaml, I also define: stacks/catalog_prod/component/XXX.yaml as:
Copy code
import:
  - <http://github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref=v1.0.0|github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref=v1.0.0>  # ⚠️ TODO: use variable for git ref. unsupported yet

components:
  terraform:
    XXX/base:
      source:
        uri: <http://github.com/my-org/my-repo//components/terraform/XXX|github.com/my-org/my-repo//components/terraform/XXX>
        version: "{{ .settings.context.deployment_repo_version }}"
Those prod_components are the components imported by prod stacks. The only remaining bit I'd like fixed is the hardcoded "v1.0.0" ref in the import line. I'd like to be able to use the same variable I use for the component source version, which is defined once, globally, for all components. Would that be possible anyhow?
e
What if...
Copy code
# dev
source:
  uri: <http://github.com/my-org/my-repo//components/terraform/XXX|github.com/my-org/my-repo//components/terraform/XXX>
  version: !git.ref
Copy code
# prod
source:
  uri: <http://github.com/my-org/my-repo//components/terraform/XXX|github.com/my-org/my-repo//components/terraform/XXX>
  version: "1.2.3"
You can define the
source.uri
once and just adjust the
version
in each stack.
We don't support
!git.ref
today, but it would be an easy add
You could use this today:
Copy code
!exec git rev-parse HEAD
a
Ohh. yes. I suppose it would result in a few more unrequired calls to github, and would deserve a comment in the dev part or catalog for people to understand what's going on but yes. Let me try this.
e
In any case, this will be in 1.221.0; it has other benefits as well https://github.com/cloudposse/atmos/pull/2528
a
what if ...
Actually, after rethinking about it a bit, no, I think it would not work: My solution allows to isolate the prod from any dev changes in both • local catalog (using the prod_catalog which imports from a versioned catalog file) • local components (using a versioned component with JIT vendoring) What you suggest, if I'm correct, only isolates from dev local component changes.
e
Something more like: (pseudo code)
Copy code
#dev
import:
  - stacks/catalog/XXX.yaml

components:
  terraform:
    XXX/base:
      source:
        uri: ./components/terraform/XXX
and prod
Copy code
import:
  - <http://github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref=v1.0.0|github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref=v1.0.0>  # ⚠️ TODO: use variable for git ref. unsupported yet

components:
  terraform:
    XXX/base:
      source:
        uri: <http://github.com/my-org/my-repo//components/terraform/XXX|github.com/my-org/my-repo//components/terraform/XXX>
        version: "{{ .settings.context.deployment_repo_version }}"
a
Exactly, this is what I did. Your dev snippet is my catalog file, and the prod snippet is my prod_catalog file. Then, my dev stacks use the catalog files, and the prod stacks use the prod_catalog files. ==> Hence the will to be able to use variable substitution in the import line:
Copy code
import:
  - "<http://github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref={{|github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref={{> .settings.context.deployment_repo_version }}"  # ⚠️ KO
e
And you are attempting to assign the deployment repo version in the global scope of settings for a particular stack?
a
exactly. I have a level common to all stacks of a given stage where I defined it.
Meaning, each import line would have to be evaluated using all variables defined in any previous import line. My stack looks like:
Copy code
import:
  - ./_defaults         # .settings.context.deployment_repo_version defined here or in something else imported from this
  - catalog_prod/XXX    # .settings.context.deployment_repo_version used here

components:
  terraform:
    XXX:
      metadata:
        inherits:
          - XXX/base
      any customized setting...
To be fair and reliable, I suppose if this feature is implemented, even this should be supported:
Copy code
import:
  - ./_defaults         # .settings.context.deployment_repo_version defined here or in something else imported from this
  - "<http://github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref={{|github.com/my-org/my-repo//stacks/catalog/XXX.yaml?ref={{> .settings.context.deployment_repo_version }}"
e
Can you update your example with where deployment repo version is set
a
Here's a stripped version of my demo repo.
deployment_repo_version
is defined in stacks/deploy/prod/_defaults.yaml, and used in stacks/catalog_prod/*
By the way, on a totally unrelated note: locals are still buggy. For example, defining any locals in stacks/deploy/_defaults.yaml results in a failure with this error:
Error: invalid stack manifest: template: mixins/atmos-pro.yaml:6:21: executing "mixins/atmos-pro.yaml" at <.atmos_component>: map has no entry for key "atmos_component"
. Locals would be nice, I have use cases for them but so far, any attempt on my side to use them led to failures.
e
Yes, locals are still wonky. We haven't had a chance to get back to it; there are some bandaids we can add, but doing it the right way will take a bigger refactor
Thanks for the demo repo, I will take a look
@Alexandre Feblot can you give this a spin with
--use-version=2554
and LMK how it goes
a
@Erik Osterman (Cloud Posse), awesome, it works great, thanks 🙂 Note: the atmos version contraint still fails when using
--use-version=xxxx
(Error: invalid version constraint ~ invalid current version "test": malformed version: test). If my memory serves right, fixing this had been at least started, but maybe that PR hasn't been merged yet (?)
e
Ok, will get this into the next release
Also thanks for re-reporting the issue with use version, will take another look
❤️ 1