<@U02BF49S8UR> hi, thanks for sharing the feedback...
# atmos-dev
a
@Dan Miller (Cloud Posse) hi, thanks for sharing the feedback on docs generate. Have looked through the comments and briefly discussed wtih @Erik Osterman (Cloud Posse), I guess the minor ones (logs, idents, admonition etc) are easy to fix, and there's one items that might required some clarification at your end. Could you please take a look at #3 below ? 1. atmos generate generate issue is due to the way cobra handles aliases. Agreed with Erik to take the alias out for now. 2. mutiple targets - could be done, in this case instead of a hardcoded
readme
subcommand we make it an argument, this will allow to have multiple targets. So we could have
readme
, or
component1
or
component2
virtually anything as an argumant to
docs genrate
as well as section in atmos.yaml. The bigger challenge here is how to document it properly so people aren't confused. 3. On defining the input with stack config. The way it currently works: we have a template and a readme.yaaml with a pretty much the same set of keys. So gomplate runs throguh the template and puts the values from the yaml file. In the proposed yaml you list docs and readme and template name sections. They are currntly taken from the
atmos.yaml
and in your proposaal you suggest to put them into the reame.yaml? Could you please elaborate a little bit what kind of problem you are trying to address?
Copy code
docs:
  readme:
    name: "template"
    # Canonical GitHub repo
    github_repo: "cloudposse-terraform-components/template"
    # Short description of this project
    description: |-
      Description of this component
    usage: |-
      **Stack Level**: Regional or Test47

      Here's an example snippet for how to use this component.
Copy code
yaml
      components:
        terraform:
          foo:
            vars:
              enabled: true
      
    include:
      - "docs/terraform.md"
    tags:
      - terraform
      - terraform-modules
      - aws
      - components
      - terraform-components
      - root
      - geodesic
      - reference-implementation
      - reference-architecture
    # Categories of this project
    categories:
      - terraform-modules/root
      - terraform-components
    # License of this project
    license: "APACHE2"
    # Badges to display
    badges:
      - name: "Latest Release"
        image: "https://img.shields.io/github/release/cloudposse-terraform-components/template.svg?style=for-the-badge"
        url: "https://github.com/cloudposse-terraform-components/template/releases/latest"
      - name: "Slack Community"
        image: "https://slack.cloudposse.com/for-the-badge.svg"
        url: "https://slack.cloudposse.com"
    references:
      - name: "Cloud Posse Documentation"
        description: "Complete documentation for the Cloud Posse solution"
        url: "https://docs.cloudposse.com"
      - name: "Reference Architectures"
        description: "Launch effortlessly with our turnkey reference architectures, built either by your team or ours."
        url: "https://cloudposse.com/"
    related:
    - name: "Cloud Posse Terraform Modules"
      description: Our collection of reusable Terraform modules used by our reference architectures.
      url: "https://docs.cloudposse.com/modules/"
    - name: "Atmos"
      description: "Atmos is like docker-compose but for your infrastructure"
      url: "https://atmos.tools"
    contributors: [] # If included generates contribs
4. atmos generate docs to build all docs options - ok, this can be done
d
Hi @Andrey Kalmykov thanks for replying! 1 and 4 sound great. However for 2 and 3, let me discuss with Erik on a call this afternoon. I have a vision in mind that I'd like to talk through
πŸ™Œ 1
e
> instead of a hardcoded
readme
subcommand we
readme
is not hardcoded, right? That's based on the dynamic configuration in the atmos config.
E.g. it could have been
foobar
a
@Erik Osterman (Cloud Posse) thanks, right now, I'm afraid it is hardcoded both in schema and as a cmd command. But after we implement multiple targets, I can redefine it as map[string}Docsgenerate and for corba I will make it as an argument instead of the subcommand, so we can pass pretty much anything
In case this is an argument is cobra, I guess there might be questions about how this is reflected in help. As of right now it is hardcocdded, with argument it will no longer be the case.
e
The intent is this was configurable.
readme
was not a well-thoughtout command name, because it was arbitrary.
a
sure, so it is ok for readme to be an argument?
@Dan Miller (Cloud Posse) hi, wanted to check if you had a chance to discuss internally the design/vision with @Erik Osterman (Cloud Posse) ?
d
hi @Andrey Kalmykov sorry for the delay! Yes we discussed, and we decided to leave the implementation as you had it. That includes a few patterns: 1. Keep docs configuration in Atmos config 2. Change the
readme
arg for
docs.generate
to a dynamic value. I believe this is the only change 3. Keep using a list for
docs.generate.readme.inputs
. This list should support remote file sources, local file sources, and maps of values, which should be merged in order
πŸ™Œ 1
however regarding #3, @Erik Osterman (Cloud Posse) where did we land on how to specify remote or local file? Option 1: (I believe this is what we have now)
Copy code
docs:
  generate:
    readme:
      inputs:
      - https://raw.githubusercontent.com/cloudposse-terraform-components/.github/main/templates/default_inputs.yaml
      - "./default_tags.yaml" 
      - name: "kinesis-firehose-stream"
        github_repo: "cloudposse-terraform-components/aws-kinesis-firehose-stream"
Option 2: (require using
!include
for consistency)
Copy code
docs:
  generate:
    readme:
      inputs:
      - !include https://raw.githubusercontent.com/cloudposse-terraform-components/.github/main/templates/default_inputs.yaml
      - !include "./default_tags.yaml" 
      - name: "kinesis-firehose-stream"
        github_repo: "cloudposse-terraform-components/aws-kinesis-firehose-stream"
a
hi, @Dan Miller (Cloud Posse), thank you so much for your inputs. I will follow through with them. Regarding the include Im not 100% sure that docs generate resolving path logic follows exactly the same logic as the !include function as its been a while. Let me check that.
πŸ‘ 1
e
I think we can git rid of suport for local and remote files, and lean entirely on !include. However, I think there's another PR that is open by @Haitham implementing that
!include
h
@Erik Osterman (Cloud Posse) yes I will add
!include
to the current PR
πŸ™Œ 1
a
@Erik Osterman (Cloud Posse) That's a great news! So once @Haitham adds !include handling to config file loader, I can use that and get rid of my custom loader for local/remote files. Gues its both for input yaml files and templates? Meanwhile I just do the one #2 change form above list.
e
Gues its both for input yaml files and templates? Meanwhile I just do the one #2 change form above list.
yes, but the one question lingering in my mind is relative paths
We solved that in the way you do it, but the relative paths for
!include
will be different
a
Oh, that’s a problem then
@Erik Osterman (Cloud Posse), hi, I ws thinking about it and , we can actually try to write some unified path resolver that does one thing if its manifest, and another if its docs but wouldn that overcomplicate things?
e
Not sure yet. I'm thinking we first get @Haitham’s PR merged for this
Then see how it works
And adjust course after that
πŸ‘ 1
a
@Erik Osterman (Cloud Posse) @Dan Miller (Cloud Posse) hi, have made the updates as discussed to https://github.com/cloudposse/atmos/pull/934 Could you please review and let me know if it is better now?
e
@Dan Miller (Cloud Posse) ☝️
πŸ‘€ 1
d
I'll take a look this afternoon. thanks!
πŸ‘ 1
I'm having trouble getting the command to work locally, but it might be missing something simple. Here's the message I keep seeing
Copy code
ERRO failed to render template with datasources: failed to render template: failed to render template docs-generate: template: docs-generate:1:5: executing "docs-generate" at <.name>: map has no entry for key "name"
Yet I have
name
defined in inputs. Here's a PR that I'm using to test out the command: https://github.com/cloudposse-terraform-components/aws-kinesis-firehose-stream/pull/9 @Andrey Kalmykov do you have any ideas of what I could be missing?
πŸ‘€ 1
ah I found the issue. It needed to be
settings.docs.generate
not
docs.generate
(I was missing
settings
)
e
You have the right location though Dan. The problem is that it’s using settings.
Settings for global things
Otherwise, the convention is that each top level command has its own top level section in the Atmos configuration
Since the top level command is docs generate, the corresponding config section should be the same
d
ah okay I see. @Andrey Kalmykov could you also add each input type to the tests? Remote, local, or map
πŸ‘ 1
e
So for example, we have a separate setting for terminals. The terminal is a global thing that’s why it’s settings.terminal.
a
hi, @Erik Osterman (Cloud Posse), so I have just added Generate section to existing Docs struct that are in the settings struct. So should I move the docs struct out of settings on-to the topl-level?
e
but what is in the "existing docs struct"? My recollection is it's 1 deprecated parameter that was moved to
settings.terminal
.
Yes, there should be a top-level
docs.generate
section
a
@Erik Osterman (Cloud Posse) sure, np, and @Dan Miller (Cloud Posse) you mean acceptance tests?
d
yes in the acceptance tests please
πŸ‘ 1
a
@Dan Miller (Cloud Posse), hi, in regards to your comment in PR with map. As of right now I have only strings in input field, i.e. local or remote files. In regars to maps, it fetches the files and subsequnetly merges it into final map using native Atmos merge fucntion logic if there are several inputs. Do you want it to be able to accept maps as well directly in config?
d
yes please. we need to accept maps directly in config like this:
Copy code
input:
        - "https://www.../README.yaml" # remote file
        - "./README.yaml" # local file
        - name: "test"    # inline map
πŸ‘ 1
a
hi @Erik Osterman (Cloud Posse), @Dan Miller (Cloud Posse), just added everything as requested(inline maps + accepatnce test + unit test). Please check if its better now.
πŸ‘€ 1
d
@Andrey Kalmykov are the list items under
input
merged in order?
I see you already added a test for this! it's a mistake on my part... checking
πŸ™Œ 1
this is working great for me now! I'll let @Erik Osterman (Cloud Posse) take another look, but otherwise LGTM!
πŸ™Œ 1
left a few minor comments on the docs page
πŸ™Œ 1
@Erik Osterman (Cloud Posse) I've review the terraform-docs feature too, and it's working as well
e
So you think we're good to merge?
the docs generate pr
d
there are a few minor things on the docs page, but I can update that
a
@Dan Miller (Cloud Posse) hi, noticed you made some good updates to the docs, are you going to finalize those or you need any help?
Let me know what’s best.
d
I'll take over the docs finalization, thanks! It'll depend on another PR to generate screengrabs, but we'll get that handled separately
πŸŽ‰ 1