<@U02JS8EFMMJ> <@UQZG2UCF3> in helpdesk we are hav...
# tools
j
@Júlia Mir @mashehu in helpdesk we are having more people with problems trying to follow the blog post w.r.t the generating the nf-test snapshots: The
nf-test test
command to generate the snapshot is misleading: • It will trigger tests for every single module too (unwanted) • There is not a default container system profile, so it first fails for missing that • Instinctively people will add
--profile docker
to the nf-test command, but this fails as it overwrites all over test configs We suggest:
Copy code
nf-test test --tag pipeline --profile=+<docker/singularity/conda etc.>
Where
--tag pipeline
makes sure you only run the
default.nf.test
test, and
--profile=+docker
specifies to run docker but in an additive manner (so does not overwrites any test.configs that are being used)
m
or I would do:
Copy code
nf-test test tests/ --profile=+<docker/singularity/conda etc.>
👍 1
👍🏻 1
m
Good point. Could you make a pr to the blog post?
salute 1
j
Also What should be we specifying then in terms of the branch protection and required tests?
Ooof and is it expected that the nf
nf-test.yml
runs EVERY SINGLE test in teh entire pipeline?!
m
Branch protection should stay the same atm (so only linting is required). That's why we have runs-on runners to handle the runner load (or do you mean also the modules tests?)
j
Ah, I had it that tests also had to pass 😬, so maybe that was a custom me thing
And for the nf-test thing: when I check my CI runs on the template merge, they are failing as they are triggering every single test in the pipeline, including modules tests. Some of these will fail as I've not installed those module tests' dependencies
Or should I do that?
m
Yep. We had a short period where we thought this would be good but we kept changing the name of them too much and kept on breaking the branch protection rules.
👍 1
j
OK I'll deactivate them
Shame though, it is was nice
m
No, the modules should not run.
Once the dust has settled my plan is to require confirm-pass like in modules.
🎉 1
j
Nice, ok that sounds good
I'll try and investigate the all module tets runnig thing
(also why does
ipeline_if_empty_null
take such a long time during linting?
quick look 1
No, the modules should not run.
I don't think there is any part in
nf-test.yml
or
action.yml
where you limit to just e.g. pipeline tags
I assume it would go here:
Copy code
- name: Run nf-test
        uses: ./.github/actions/nf-test
        env:
          NFT_DIFF: ${{ env.NFT_DIFF }}
          NFT_DIFF_ARGS: ${{ env.NFT_DIFF_ARGS }}
          NFT_WORKDIR: ${{ env.NFT_WORKDIR }}
        with:
          profile: ${{ matrix.profile }}
          shard: ${{ matrix.shard }}
          total_shards: ${{ env.TOTAL_SHARDS }}
Aaaand might have some self-hosted runner issues
Copy code
>   Loading ncbi.tre: OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fd266000000, 16877879296, 0) failed; error='Not enough space' (errno=12)
    >   #
    >   # There is insufficient memory for the Java Runtime Environment to continue.
    >   # Native memory allocation (mmap) failed to map 16877879296 bytes for committing reserved memory.
    >   # An error report file with more information is saved as:
    >   # hs_err_pid33.log
@Júlia Mir @mashehu maybe you need to add to your release checklist 'Get James to try a template merge on one of his stupid pipelines to fine all the stupid edge cases'
Oh no wait the noit enough space might be a random thing
(also why does ipeline_if_empty_null take such a long time during linting?
This appears tobe only locally though
m
make an issue please for the slow lint test.
👍 1
@Maxime Garcia i think you wrote the testing logic. modules tests shouldn’t run, right?
m
yeah, I think we should change this logic to:
Copy code
ignore 'modules/nf-core/**/tests', 'subworkflows/nf-core/**/tests'
j
This is after that though
So I don't think that worked
m
because it actually ignores the scripts coming from nf-core, not only the tests
j
What worked for me was running only on tags with
pipeline
m
I usually run things with the path to the tests
but tags should work as well
j
I went with tags because the action itself supports that
m
there might be a patch release upcoming. so would be good if we get that settled until then 🙂
m
oh
Let me push what I wanted to do then on the nft-utils side
I can do it
Change there on line 109
m
meh
If you do that you don't run any local tests you might have implemented
ie tests for local modules + local subworkflows
j
What do you mean?
m
so I'm against it
j
Ahhhhh
I see now
But I mean I don't want ot run every single local module test every time I run a whole pipeline update
pipeline test run*
m
that would be taken care of
nf-test-detect-changes
no?
party this 1
j
Should do... OK let me remove my tag limit and see what happens
For the nf-test-detect-changes though should I expect them all to run the very first time?
m
yes because of the files you modify in this PR
Copy code
// list of filenames or patterns that should be trigger a full test run
    triggers 'nextflow.config', 'nf-test.config', 'conf/test.config', 'tests/nextflow.config', 'tests/.nftignore'
j
But I've not modified that file in this PR
Ahh
m
changing these files will trigger all tests
j
I seee
Guys, we need to document these things 😅
Describe the logic/behaviour etc
m
on my list
m
Things are documented, I would say we need to improve the documentation
j
Things are documented, I would say we need to improve the documentation
Where?
m
ci test and logic isn’t really documented. (sharding e.g. is only mentioned in the blogpost)
j
But I mean before release (except this time of course, because the royal WE needs it for next week 😉 )
m
Where?
Comments are documentation
👎🏻 1
👎 1
🚷 1
shake no 1
j
Good descriptive comments yes 😉
They should describe why things
Copy code
// ignore tests coming from the nf-core/modules repo
Is not why
From the tidyerse god:
Comment your code. Each line of a comment should begin with the comment symbol and a single space: #. Comments should explain the why, not the what.
http://adv-r.had.co.nz/Style.html
And @Maxime Garcia this line is not working:
Copy code
ignore 'modules/nf-core/**/tests', 'subworkflows/nf-core/**/tests'
Or have I made a typo?
m
ok sorry about my lack of documentation
j
Don't worry. I will rage at the retreat at ALL OF YOU 😘
m
And @Maxime Garcia this line is not working:
How is this not working?
j
It's running all module tests for me
image.png
I'm trying:
Copy code
ignore 'modules/nf-core/**/tests/*', 'subworkflows/nf-core/**/tests/*'
now
m
ok
j
That appears to be working
m
oooh
Nice
or
tests/
?
j
That idon't know
m
ok, nevermind, it's a good improvment already
j
??
Also was there a fix for pdiff in GHA?
I'm getting a subworkflow failure with no info
(locally working fine)
m
yes
Let me find this
❤️ 1
j
OK, was able to replicate on codespaces for now
(the fialure)
j
Hrm, IO did that
I thjought
Ah missed the second one
m
let me know if it works
m
okay, any PR-able conclusions here?
j
Add
/tests
dir at the end of the ignore list
and document to expect EVERYTHING to get run the first time