<@U052RGXL350> Hello. Thanks for your question. Le...
# github-actions
i
@hatz Hello. Thanks for your question. Let me try to reproduce the issue
h
Awesome! I did end up doing a whole lot of testing and I do have it all working now, but there are some odd things. The first big fix I found wasx just throwing quotes around things, like this:
Copy code
outputs: |
          url: "${{ env.URL }}"
which seems to fix the
-
issue because that character in yaml is technically for an array
Another odd thing I found is that an individual key doesn't like to be given multiline yaml. For example if I have this input in my github action
Copy code
paths:
    description: 'A multiline yaml string of key/value pairs for each Name and Path to include in the comment'
    required: false
    default: |
      Application: application
      Dashboard: /dashboard
And then try to directly use it to the matrix write action like
Copy code
paths: ${{ inputs.paths }}
It ends up breaking because of the newlines and a similar map issue. I got around this currently by turning my multiline yaml into a singleline yaml with newline characters, passing that into the write, and then turning it back into a multiline yaml on the other side. for example
Copy code
- name: Process paths for output
      shell: bash
      run: |
        echo "paths=$(echo '${{ inputs.paths }}' | awk '{printf "%s\\n", $0}')" >> $GITHUB_ENV

    - uses: cloudposse/github-action-matrix-outputs-write@0.3.1
      id: out
      with:
        matrix-step-name: ${{ github.job }}
        matrix-key: ${{ env.APPLICATION_NAME }}
        outputs: |
          url: "${{ env.URL }}"
          paths: '${{ env.paths }}'
And then on the read side doing a
echo -e
so that the newlines get expanded. Happy to share more details with you if you'd like! Passing around the multiline yaml stuff gets annoying complicated in github actions I've found so this might be some edge cases šŸ˜„
i
@hatz I create a PR with new test for your case
Could you check that I get the issue right ?
h
Also I'm not sure how much it matters but I am currently using the yaml
: |
instead of the
: |-
- not sure if the newline at the end breaks anything or not but thought I'd mention it! example:
Copy code
outputs: |
          url: '${{ env.URL }}'
          paths: '${{ env.paths }}'
i
@hatz I’m rewriting the action to nodejs
otherwise it would be always edge cases that breaks everything
h
That is awesome! I'm currently building some internal composite actions for our engineers to do ephemeral deployments, and doing stuff straight in bash has been kind of a headache because of newlines and special characters and all that.
Shell gets brittle really fast when trying to do multiline stuff I've found. And github actions seems to behave slightly different than my own shell so it's been a lot of experimenting šŸ˜‚
If you want any help or need someone to do any kind of testing let me know! I've got my side working (with some tweaks to pre/post-process) it with 0.3.1 and definitely happy to test new stuff!
I cut the new release
Could you pls test your special chars issues?
for multiline unfortunately you have to use such workaround
Copy code
outputs: |
          url: '${{ env.URL }}'
          paths: ${{ toJson(env.paths) }}
this is the limitation of GitHub actions substitution.
h
Hey! I haven't had a chance to test this out yet, been OOO for a few days. I will probably get this tomorrow or friday!
šŸ‘ 1
Hey @Igor Rodionov - My first initial test led me to believe this didn't work but I'm trying a couple other things. It downloaded all 9 of the files that I expected to see, but the 'output' only had one entry for each object. Possible I might have something on my side so I'll let you know!
Alright one error I'm finding so far is what it considers to be valid yaml.
Copy code
paths: 'Application:
'
fd_name: 'frontend-asset-manager'
---------------------
YAMLParseError: Missing closing 'quote at line 2, column 21:

paths: 'Application:
                    ^
This seems to be related to the multi-line yaml that I have where the value is an empty string here is my input
Copy code
outputs: |
          url: ${{ env.URL }}
          paths: '${{ inputs.paths }}'
          fd_name: ${{ env.FD_NAME }}
I have the closing quote on the second line but it doesn't seem to see that
The new error output though is much better šŸ™‚
Here's the error I get if I don't throw single quotes around it
Copy code
Error: Outputs should be valid YAML 
---------------------
url: https://frontend-usdf-382.figureapps.dev
paths: Application:

fd_name: frontend-usdf
---------------------
YAMLParseError: Nested mappings are not allowed in compact mappings at line 2, column 8:

paths: Application:
       ^
I've tried with both
outputs: |
and
outputs: |-
too
i
Copy code
paths: '${{ toJson(inputs.paths) }}'
try
šŸ‘ 1
h
Alright this worked on the write step now:
Copy code
paths: '${{ toJson(inputs.paths) }}'
it ends up looking like this:
Copy code
with:
    matrix-step-name: ephemeral-deploy
    matrix-key: frontend-auth-382
    outputs: url: https://frontend-auth-382.url.com
      paths: '"Application:\n"'
      fd_name: frontend-auth
However, on the read side I'm still only actually seeing one entry.
Copy code
No artifact name specified, downloading all artifacts
Creating an extra directory for each artifact that is being downloaded
starting download of artifact 1ad31b158455284757e5320e8e4ca65ed4c909fce6554d4713a764e6480d9f11 : 1/9
Total number of files that will be downloaded: 1
starting download of artifact 1de3311f3a5fb2f26beb247ba1ee2e3866eac7f2cd014ca2b940117f87d46d22 : 2/9
Total number of files that will be downloaded: 1
starting download of artifact 4fc8f69a7cd20390d89bc18ea6400ec381bfea1ff6fe77c64d0b0439ab3de809 : 3/9
Total number of files that will be downloaded: 1
starting download of artifact 4fea876fd1027053d0ddf38e74b805fbe8a4e1944613e2c7aecb501d79f1d512 : 4/9
Total number of files that will be downloaded: 1
starting download of artifact 6c4186c01f171e8d190e6b8688645a0591030d0cced58c86b353d565a0211bda : 5/9
Total number of files that will be downloaded: 1
starting download of artifact 86464d497618037bd50d8e9925b69000e8e433dae9c05a2a99e356fd93147d90 : 6/9
Total number of files that will be downloaded: 1
starting download of artifact a23bd2a5d63a624994e3258b3dc50af5a26bdb6fe7ec483b51a89546f99ba47b : 7/9
Total number of files that will be downloaded: 1
starting download of artifact bf4c9fa23d050fb0912454d8a2a9d6d40f16c348289320fde9e4d9e59f21d635 : 8/9
Total number of files that will be downloaded: 1
starting download of artifact d7a846ded11eb21020889310d301d57d579499d0034000632dd9dfe1f27571c8 : 9/9
Total number of files that will be downloaded: 1
There were 9 artifacts downloaded
Everything gets downloaded. but then the json seems to not be merging them together
Copy code
json=$(echo '${{ steps.read-matrix.outputs.result }}')
        echo $json
Copy code
{
  "url": {
    "matrix_key": "https://frontend.url.com"
  },
  "paths": {
    "matrix_key": "\"Application:\\n\""
  },
  "fd_name": {
    "matrix_key": "frontend"
  }
}
also worth noting that no single quotes had a slightly different write input
Copy code
paths: ${{ toJson(inputs.paths) }}
became
Copy code
paths: "Application:\n"
but the read was the same issue
Manually inspecting all 9 of the artifacts that got uploaded during the job and I see what I expect for each one, so it seems the merging on the read somehow isn't quite working example artifact uploads (all 9 are different values but same keys)
Copy code
{
  "matrix_key": {
    "url": "https://frontend-auth.url.com",
    "paths": "Application:\n",
    "fd_name": "frontend-auth"
  }
}
Copy code
{
  "matrix_key": {
    "url": "https://frontend-group.url.com",
    "paths": "Application:\n",
    "fd_name": "frontend-group"
  }
}
Oh I see the issue. It seems like that json structure changed. I think the current version of the
-write
action is now incompatible with the
-read
action. Here's an example json from
0.3.1
release:
Copy code
{
  "frontend-auth": {
    "url": "https://frontend-auth.url.com",
    "paths": "Application:\\n\\n",
    "fd_name": "frontend-auth"
  }
}
I think the bug is that it is using the
matrix_key
as a string literal instead of a value ref
@Igor Rodionov Alright I submitted a PR for that specific issue here: https://github.com/cloudposse/github-action-matrix-outputs-write/pull/20
I'm now running into a new issue where the newline character in the json is not being evaluated properly (because in github actions I think it wants
\\n
instead of
\n
, but at least the first issue should be fixed)
Alright I fixed my issue around the newline character, in my post-processing of the read action I had some weird stuff to handle the
\\n
and I forgot to remove it in my testing, so it's all working well with the above PR fix I made + the
toJson()
you recommended!
@Igor Rodionov can you check that PR out today? I'd love to get it merged/released so we can start using it internally!
Also can you elaborate on why you do an assertion for empty fields? I have a case where passing empty fields is valid, seems like a bit of an over-reaching assertion on the input data to not allow empty string. Thoughts?
@Igor Rodionov
Copy code
with:
    matrix-step-name: ephemeral-deploy
    outputs: url: ''
  paths: 'Application:\n\n'
  fd_name: ''
  errors: '\n\n```\n:x: argocd_token is null. Please reach out to #devops so we can help you onboard.\n```'
e.g. if an error happens I'm passing
errors
through, but that means other fields (like my url) never got created so that becomes
''
Copy code
Error: Expected '' to not equal ''